Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 3,
 'device': device(type='cuda', index=0),
 'dropout': 0.30000000000000004,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2400,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 2400,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.367560
Epoch 1
Validation binary_cross_entropy = 0.466815
Epoch 2
Validation binary_cross_entropy = 0.447236
Epoch 3
Validation binary_cross_entropy = 0.256076
Epoch 4
Loss = 6.1392e-01, PNorm = 87.7990, GNorm = 3.2782, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.964445
Epoch 5
Validation binary_cross_entropy = 0.289575
Epoch 6
Validation binary_cross_entropy = 0.504531
Epoch 7
Validation binary_cross_entropy = 0.400960
Epoch 8
Validation binary_cross_entropy = 0.334300
Epoch 9
Loss = 3.8183e-01, PNorm = 88.2136, GNorm = 2.0904, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.413761
Epoch 10
Validation binary_cross_entropy = 0.380454
Epoch 11
Validation binary_cross_entropy = 0.387084
Epoch 12
Validation binary_cross_entropy = 0.502448
Epoch 13
Validation binary_cross_entropy = 0.379001
Epoch 14
Loss = 3.6630e-01, PNorm = 88.6337, GNorm = 2.1422, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.337805
Epoch 15
Validation binary_cross_entropy = 0.383700
Epoch 16
Validation binary_cross_entropy = 0.511743
Epoch 17
Validation binary_cross_entropy = 0.387309
Epoch 18
Validation binary_cross_entropy = 0.347193
Epoch 19
Loss = 3.1695e-01, PNorm = 88.9136, GNorm = 2.3440, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.359269
Epoch 20
Validation binary_cross_entropy = 0.490260
Epoch 21
Validation binary_cross_entropy = 0.449843
Epoch 22
Validation binary_cross_entropy = 0.361078
Epoch 23
Validation binary_cross_entropy = 0.356193
Epoch 24
Loss = 2.7261e-01, PNorm = 89.1232, GNorm = 1.3116, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.396858
Epoch 25
Validation binary_cross_entropy = 0.407928
Epoch 26
Validation binary_cross_entropy = 0.320517
Epoch 27
Validation binary_cross_entropy = 0.305426
Epoch 28
Validation binary_cross_entropy = 0.347792
Epoch 29
Loss = 2.1842e-01, PNorm = 89.3106, GNorm = 1.4599, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.513321
Epoch 30
Validation binary_cross_entropy = 0.347735
Epoch 31
Validation binary_cross_entropy = 0.310132
Epoch 32
Validation binary_cross_entropy = 0.346757
Epoch 33
Validation binary_cross_entropy = 0.516655
Epoch 34
Loss = 2.8963e-01, PNorm = 89.4795, GNorm = 2.7719, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.474672
Epoch 35
Validation binary_cross_entropy = 0.350820
Epoch 36
Validation binary_cross_entropy = 0.350871
Epoch 37
Validation binary_cross_entropy = 0.346396
Epoch 38
Validation binary_cross_entropy = 0.376763
Epoch 39
Loss = 2.2708e-01, PNorm = 89.7062, GNorm = 2.2309, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.430033
Epoch 40
Validation binary_cross_entropy = 0.404805
Epoch 41
Validation binary_cross_entropy = 0.346464
Epoch 42
Validation binary_cross_entropy = 0.341761
Epoch 43
Validation binary_cross_entropy = 0.347439
Epoch 44
Loss = 1.7976e-01, PNorm = 89.9015, GNorm = 0.7576, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.402011
Epoch 45
Validation binary_cross_entropy = 0.412800
Epoch 46
Validation binary_cross_entropy = 0.380495
Epoch 47
Validation binary_cross_entropy = 0.376127
Epoch 48
Validation binary_cross_entropy = 0.393183
Epoch 49
Loss = 2.3851e-01, PNorm = 90.1125, GNorm = 2.1308, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.391658
Epoch 50
Validation binary_cross_entropy = 0.359317
Epoch 51
Validation binary_cross_entropy = 0.347762
Epoch 52
Validation binary_cross_entropy = 0.355850
Epoch 53
Validation binary_cross_entropy = 0.381061
Epoch 54
Loss = 2.7084e-01, PNorm = 90.3765, GNorm = 2.2515, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.371936
Epoch 55
Validation binary_cross_entropy = 0.388689
Epoch 56
Validation binary_cross_entropy = 0.409406
Epoch 57
Validation binary_cross_entropy = 0.390362
Epoch 58
Validation binary_cross_entropy = 0.383919
Epoch 59
Loss = 2.2305e-01, PNorm = 90.7083, GNorm = 2.9754, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.399214
Epoch 60
Validation binary_cross_entropy = 0.438424
Epoch 61
Validation binary_cross_entropy = 0.446632
Epoch 62
Validation binary_cross_entropy = 0.429307
Epoch 63
Validation binary_cross_entropy = 0.421230
Epoch 64
Loss = 1.3242e-01, PNorm = 91.0120, GNorm = 2.4106, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.413092
Epoch 65
Validation binary_cross_entropy = 0.442711
Epoch 66
Validation binary_cross_entropy = 0.413554
Epoch 67
Validation binary_cross_entropy = 0.408106
Epoch 68
Validation binary_cross_entropy = 0.434504
Epoch 69
Loss = 2.0126e-01, PNorm = 91.3382, GNorm = 1.7442, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.426931
Epoch 70
Validation binary_cross_entropy = 0.454517
Epoch 71
Validation binary_cross_entropy = 0.423698
Epoch 72
Validation binary_cross_entropy = 0.402084
Epoch 73
Validation binary_cross_entropy = 0.408045
Epoch 74
Loss = 1.3095e-01, PNorm = 91.6713, GNorm = 0.9719, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.407255
Epoch 75
Validation binary_cross_entropy = 0.423070
Epoch 76
Validation binary_cross_entropy = 0.441422
Epoch 77
Validation binary_cross_entropy = 0.461727
Epoch 78
Validation binary_cross_entropy = 0.471103
Epoch 79
Loss = 9.7563e-02, PNorm = 91.9491, GNorm = 0.7809, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.479144
Epoch 80
Validation binary_cross_entropy = 0.490738
Epoch 81
Validation binary_cross_entropy = 0.506961
Epoch 82
Validation binary_cross_entropy = 0.508419
Epoch 83
Validation binary_cross_entropy = 0.501124
Epoch 84
Loss = 1.0002e-01, PNorm = 92.1896, GNorm = 0.4103, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.511520
Epoch 85
Validation binary_cross_entropy = 0.537818
Epoch 86
Validation binary_cross_entropy = 0.519839
Epoch 87
Validation binary_cross_entropy = 0.503096
Epoch 88
Validation binary_cross_entropy = 0.509110
Epoch 89
Loss = 1.1656e-01, PNorm = 92.4422, GNorm = 1.0885, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.497444
Epoch 90
Validation binary_cross_entropy = 0.515804
Epoch 91
Validation binary_cross_entropy = 0.529292
Epoch 92
Validation binary_cross_entropy = 0.499190
Epoch 93
Validation binary_cross_entropy = 0.492818
Epoch 94
Loss = 7.7186e-02, PNorm = 92.7197, GNorm = 0.8476, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.502082
Epoch 95
Validation binary_cross_entropy = 0.480918
Epoch 96
Validation binary_cross_entropy = 0.496631
Epoch 97
Validation binary_cross_entropy = 0.504578
Epoch 98
Validation binary_cross_entropy = 0.476985
Epoch 99
Loss = 8.6556e-02, PNorm = 92.9599, GNorm = 0.9716, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.471279
Epoch 100
Validation binary_cross_entropy = 0.480282
Epoch 101
Validation binary_cross_entropy = 0.491953
Epoch 102
Validation binary_cross_entropy = 0.508204
Epoch 103
Validation binary_cross_entropy = 0.534330
Epoch 104
Loss = 9.9911e-02, PNorm = 93.1867, GNorm = 1.6524, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.521677
Epoch 105
Validation binary_cross_entropy = 0.510674
Epoch 106
Validation binary_cross_entropy = 0.514131
Epoch 107
Validation binary_cross_entropy = 0.506721
Epoch 108
Validation binary_cross_entropy = 0.500047
Epoch 109
Loss = 1.3114e-01, PNorm = 93.3994, GNorm = 2.3776, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.507684
Epoch 110
Validation binary_cross_entropy = 0.536053
Epoch 111
Validation binary_cross_entropy = 0.543618
Epoch 112
Validation binary_cross_entropy = 0.528772
Epoch 113
Validation binary_cross_entropy = 0.538306
Epoch 114
Loss = 9.2873e-02, PNorm = 93.6398, GNorm = 0.4786, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.546730
Epoch 115
Validation binary_cross_entropy = 0.555413
Epoch 116
Validation binary_cross_entropy = 0.555852
Epoch 117
Validation binary_cross_entropy = 0.549498
Epoch 118
Validation binary_cross_entropy = 0.549337
Epoch 119
Loss = 4.1784e-02, PNorm = 93.8589, GNorm = 0.6014, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.562210
Epoch 120
Validation binary_cross_entropy = 0.582426
Epoch 121
Validation binary_cross_entropy = 0.600165
Epoch 122
Validation binary_cross_entropy = 0.619958
Epoch 123
Validation binary_cross_entropy = 0.626562
Epoch 124
Loss = 1.0968e-01, PNorm = 94.1249, GNorm = 1.4771, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.665382
Epoch 125
Validation binary_cross_entropy = 0.648996
Epoch 126
Validation binary_cross_entropy = 0.597174
Epoch 127
Validation binary_cross_entropy = 0.576475
Epoch 128
Validation binary_cross_entropy = 0.575137
Epoch 129
Loss = 6.5130e-02, PNorm = 94.3618, GNorm = 0.9546, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.586998
Epoch 130
Validation binary_cross_entropy = 0.588058
Epoch 131
Validation binary_cross_entropy = 0.570626
Epoch 132
Validation binary_cross_entropy = 0.584804
Epoch 133
Validation binary_cross_entropy = 0.581583
Epoch 134
Loss = 5.5548e-02, PNorm = 94.5956, GNorm = 0.7507, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.601305
Epoch 135
Validation binary_cross_entropy = 0.618489
Epoch 136
Validation binary_cross_entropy = 0.622635
Epoch 137
Validation binary_cross_entropy = 0.619101
Epoch 138
Validation binary_cross_entropy = 0.635872
Epoch 139
Loss = 3.8289e-02, PNorm = 94.8050, GNorm = 0.3850, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.663730
Epoch 140
Validation binary_cross_entropy = 0.616256
Epoch 141
Validation binary_cross_entropy = 0.633534
Epoch 142
Validation binary_cross_entropy = 0.609674
Epoch 143
Validation binary_cross_entropy = 0.551324
Epoch 144
Loss = 1.3724e-02, PNorm = 95.0088, GNorm = 0.2599, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.575627
Epoch 145
Validation binary_cross_entropy = 0.558380
Epoch 146
Validation binary_cross_entropy = 0.552018
Epoch 147
Validation binary_cross_entropy = 0.593727
Epoch 148
Validation binary_cross_entropy = 0.625785
Epoch 149
Loss = 9.3931e-02, PNorm = 95.2431, GNorm = 1.7503, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.619261
Epoch 150
Validation binary_cross_entropy = 0.605604
Epoch 151
Validation binary_cross_entropy = 0.606273
Epoch 152
Validation binary_cross_entropy = 0.606188
Epoch 153
Validation binary_cross_entropy = 0.606136
Epoch 154
Loss = 9.4018e-02, PNorm = 95.4451, GNorm = 0.6266, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.635866
Epoch 155
Validation binary_cross_entropy = 0.692982
Epoch 156
Validation binary_cross_entropy = 0.659979
Epoch 157
Validation binary_cross_entropy = 0.619228
Epoch 158
Validation binary_cross_entropy = 0.627095
Epoch 159
Loss = 6.0751e-02, PNorm = 95.6581, GNorm = 0.7521, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.643813
Epoch 160
Validation binary_cross_entropy = 0.671855
Epoch 161
Validation binary_cross_entropy = 0.710355
Epoch 162
Validation binary_cross_entropy = 0.706513
Epoch 163
Validation binary_cross_entropy = 0.653030
Epoch 164
Loss = 5.0123e-02, PNorm = 95.8362, GNorm = 0.7660, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.606519
Epoch 165
Validation binary_cross_entropy = 0.571327
Epoch 166
Validation binary_cross_entropy = 0.543895
Epoch 167
Validation binary_cross_entropy = 0.534583
Epoch 168
Validation binary_cross_entropy = 0.538488
Epoch 169
Loss = 3.7579e-02, PNorm = 96.0122, GNorm = 0.7239, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.551373
Epoch 170
Validation binary_cross_entropy = 0.564296
Epoch 171
Validation binary_cross_entropy = 0.574637
Epoch 172
Validation binary_cross_entropy = 0.581925
Epoch 173
Validation binary_cross_entropy = 0.587978
Epoch 174
Loss = 7.9389e-02, PNorm = 96.1652, GNorm = 0.4658, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.593788
Epoch 175
Validation binary_cross_entropy = 0.600846
Epoch 176
Validation binary_cross_entropy = 0.608685
Epoch 177
Validation binary_cross_entropy = 0.614252
Epoch 178
Validation binary_cross_entropy = 0.628483
Epoch 179
Loss = 3.8827e-02, PNorm = 96.2928, GNorm = 0.7052, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.649608
Epoch 180
Validation binary_cross_entropy = 0.686337
Epoch 181
Validation binary_cross_entropy = 0.719905
Epoch 182
Validation binary_cross_entropy = 0.752182
Epoch 183
Validation binary_cross_entropy = 0.764856
Epoch 184
Loss = 2.0008e-02, PNorm = 96.4376, GNorm = 0.3774, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.764140
Epoch 185
Validation binary_cross_entropy = 0.761989
Epoch 186
Validation binary_cross_entropy = 0.768056
Epoch 187
Validation binary_cross_entropy = 0.750705
Epoch 188
Validation binary_cross_entropy = 0.718524
Epoch 189
Loss = 4.1115e-02, PNorm = 96.5979, GNorm = 0.4982, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.706819
Epoch 190
Validation binary_cross_entropy = 0.714981
Epoch 191
Validation binary_cross_entropy = 0.741258
Epoch 192
Validation binary_cross_entropy = 0.738200
Epoch 193
Validation binary_cross_entropy = 0.738847
Epoch 194
Loss = 2.0766e-02, PNorm = 96.7509, GNorm = 0.3076, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.755363
Epoch 195
Validation binary_cross_entropy = 0.784393
Epoch 196
Validation binary_cross_entropy = 0.824351
Epoch 197
Validation binary_cross_entropy = 0.861489
Epoch 198
Validation binary_cross_entropy = 0.795920
Epoch 199
Loss = 2.3060e-02, PNorm = 96.8999, GNorm = 0.1335, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.769084
Epoch 200
Validation binary_cross_entropy = 0.771370
Epoch 201
Validation binary_cross_entropy = 0.808491
Epoch 202
Validation binary_cross_entropy = 0.842778
Epoch 203
Validation binary_cross_entropy = 0.863012
Epoch 204
Loss = 2.5539e-02, PNorm = 97.0441, GNorm = 0.9106, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.868598
Epoch 205
Validation binary_cross_entropy = 0.860778
Epoch 206
Validation binary_cross_entropy = 0.860553
Epoch 207
Validation binary_cross_entropy = 0.864718
Epoch 208
Validation binary_cross_entropy = 0.851804
Epoch 209
Loss = 1.6030e-02, PNorm = 97.1682, GNorm = 0.5451, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.836837
Epoch 210
Validation binary_cross_entropy = 0.835070
Epoch 211
Validation binary_cross_entropy = 0.835533
Epoch 212
Validation binary_cross_entropy = 0.839169
Epoch 213
Validation binary_cross_entropy = 0.845095
Epoch 214
Loss = 3.0897e-02, PNorm = 97.3242, GNorm = 1.4899, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.875160
Epoch 215
Validation binary_cross_entropy = 0.902260
Epoch 216
Validation binary_cross_entropy = 0.926202
Epoch 217
Validation binary_cross_entropy = 0.947091
Epoch 218
Validation binary_cross_entropy = 0.981526
Epoch 219
Loss = 6.2293e-02, PNorm = 97.4899, GNorm = 1.0204, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.909602
Epoch 220
Validation binary_cross_entropy = 0.839355
Epoch 221
Validation binary_cross_entropy = 0.814331
Epoch 222
Validation binary_cross_entropy = 0.803509
Epoch 223
Validation binary_cross_entropy = 0.803184
Epoch 224
Loss = 3.2933e-02, PNorm = 97.7155, GNorm = 0.2554, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.813028
Epoch 225
Validation binary_cross_entropy = 0.773963
Epoch 226
Validation binary_cross_entropy = 0.747718
Epoch 227
Validation binary_cross_entropy = 0.733047
Epoch 228
Validation binary_cross_entropy = 0.723045
Epoch 229
Loss = 1.5776e-02, PNorm = 97.9172, GNorm = 0.2257, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.730417
Epoch 230
Validation binary_cross_entropy = 0.757807
Epoch 231
Validation binary_cross_entropy = 0.815477
Epoch 232
Validation binary_cross_entropy = 0.846262
Epoch 233
Validation binary_cross_entropy = 0.873413
Epoch 234
Loss = 2.0289e-02, PNorm = 98.1135, GNorm = 0.4919, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.884093
Epoch 235
Validation binary_cross_entropy = 0.892221
Epoch 236
Validation binary_cross_entropy = 0.912116
Epoch 237
Validation binary_cross_entropy = 0.953701
Epoch 238
Validation binary_cross_entropy = 0.971521
Epoch 239
Loss = 5.2805e-02, PNorm = 98.2639, GNorm = 0.0743, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.980157
Epoch 240
Validation binary_cross_entropy = 0.978563
Epoch 241
Validation binary_cross_entropy = 0.990858
Epoch 242
Validation binary_cross_entropy = 1.004908
Epoch 243
Validation binary_cross_entropy = 1.017834
Epoch 244
Loss = 6.7786e-03, PNorm = 98.3783, GNorm = 0.0764, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.027479
Epoch 245
Validation binary_cross_entropy = 1.029859
Epoch 246
Validation binary_cross_entropy = 1.045775
Epoch 247
Validation binary_cross_entropy = 1.057932
Epoch 248
Validation binary_cross_entropy = 1.046200
Epoch 249
Loss = 7.5967e-03, PNorm = 98.4694, GNorm = 0.1806, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.991083
Epoch 250
Validation binary_cross_entropy = 0.948568
Epoch 251
Validation binary_cross_entropy = 0.916322
Epoch 252
Validation binary_cross_entropy = 0.902877
Epoch 253
Validation binary_cross_entropy = 0.902528
Epoch 254
Loss = 5.3710e-03, PNorm = 98.5578, GNorm = 0.1548, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.905381
Epoch 255
Validation binary_cross_entropy = 0.914304
Epoch 256
Validation binary_cross_entropy = 1.000359
Epoch 257
Validation binary_cross_entropy = 0.980453
Epoch 258
Validation binary_cross_entropy = 0.966839
Epoch 259
Loss = 4.3913e-02, PNorm = 98.6561, GNorm = 1.6682, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.973805
Epoch 260
Validation binary_cross_entropy = 1.026068
Epoch 261
Validation binary_cross_entropy = 1.079391
Epoch 262
Validation binary_cross_entropy = 1.095723
Epoch 263
Validation binary_cross_entropy = 1.067318
Epoch 264
Loss = 1.8458e-02, PNorm = 98.7866, GNorm = 0.2411, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.017304
Epoch 265
Validation binary_cross_entropy = 0.988557
Epoch 266
Validation binary_cross_entropy = 0.980001
Epoch 267
Validation binary_cross_entropy = 0.952394
Epoch 268
Validation binary_cross_entropy = 0.915377
Epoch 269
Loss = 2.9325e-03, PNorm = 98.9030, GNorm = 0.0557, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.887895
Epoch 270
Validation binary_cross_entropy = 0.865233
Epoch 271
Validation binary_cross_entropy = 0.835824
Epoch 272
Validation binary_cross_entropy = 0.817760
Epoch 273
Validation binary_cross_entropy = 0.833069
Epoch 274
Loss = 8.6453e-03, PNorm = 99.0165, GNorm = 0.3969, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.866010
Epoch 275
Validation binary_cross_entropy = 0.864679
Epoch 276
Validation binary_cross_entropy = 0.866869
Epoch 277
Validation binary_cross_entropy = 0.886499
Epoch 278
Validation binary_cross_entropy = 0.915926
Epoch 279
Loss = 7.5670e-03, PNorm = 99.0947, GNorm = 0.0994, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.940879
Epoch 280
Validation binary_cross_entropy = 0.948193
Epoch 281
Validation binary_cross_entropy = 0.944644
Epoch 282
Validation binary_cross_entropy = 0.937628
Epoch 283
Validation binary_cross_entropy = 0.944430
Epoch 284
Loss = 5.2540e-02, PNorm = 99.1853, GNorm = 1.8908, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.969646
Epoch 285
Validation binary_cross_entropy = 1.008245
Epoch 286
Validation binary_cross_entropy = 1.006626
Epoch 287
Validation binary_cross_entropy = 0.935790
Epoch 288
Validation binary_cross_entropy = 0.866533
Epoch 289
Loss = 6.2968e-03, PNorm = 99.3321, GNorm = 0.0846, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.838411
Epoch 290
Validation binary_cross_entropy = 0.834397
Epoch 291
Validation binary_cross_entropy = 0.826163
Epoch 292
Validation binary_cross_entropy = 0.845749
Epoch 293
Validation binary_cross_entropy = 0.881689
Epoch 294
Loss = 1.3008e-02, PNorm = 99.4750, GNorm = 0.2738, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.904203
Epoch 295
Validation binary_cross_entropy = 0.919563
Epoch 296
Validation binary_cross_entropy = 0.926858
Epoch 297
Validation binary_cross_entropy = 0.932437
Epoch 298
Validation binary_cross_entropy = 0.941080
Epoch 299
Loss = 1.2075e-02, PNorm = 99.5936, GNorm = 0.2887, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.957102
Model 0 best validation binary_cross_entropy = 0.256076 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.173575
Ensemble test binary_cross_entropy = 0.173575
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.370914
Epoch 1
Validation binary_cross_entropy = 0.719939
Epoch 2
Validation binary_cross_entropy = 0.344685
Epoch 3
Validation binary_cross_entropy = 0.610512
Epoch 4
Loss = 4.8408e-01, PNorm = 87.7980, GNorm = 3.8870, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.456861
Epoch 5
Validation binary_cross_entropy = 0.399883
Epoch 6
Validation binary_cross_entropy = 0.385061
Epoch 7
Validation binary_cross_entropy = 0.464678
Epoch 8
Validation binary_cross_entropy = 0.389088
Epoch 9
Loss = 5.2689e-01, PNorm = 88.2232, GNorm = 3.0411, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.319484
Epoch 10
Validation binary_cross_entropy = 0.394049
Epoch 11
Validation binary_cross_entropy = 0.369089
Epoch 12
Validation binary_cross_entropy = 0.329094
Epoch 13
Validation binary_cross_entropy = 0.370623
Epoch 14
Loss = 3.4327e-01, PNorm = 88.6652, GNorm = 1.6908, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.310109
Epoch 15
Validation binary_cross_entropy = 0.323838
Epoch 16
Validation binary_cross_entropy = 0.409655
Epoch 17
Validation binary_cross_entropy = 0.336330
Epoch 18
Validation binary_cross_entropy = 0.331749
Epoch 19
Loss = 3.1928e-01, PNorm = 88.9994, GNorm = 2.0893, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.555088
Epoch 20
Validation binary_cross_entropy = 0.385213
Epoch 21
Validation binary_cross_entropy = 0.322880
Epoch 22
Validation binary_cross_entropy = 0.351248
Epoch 23
Validation binary_cross_entropy = 0.436453
Epoch 24
Loss = 2.9348e-01, PNorm = 89.2671, GNorm = 2.5914, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.348028
Epoch 25
Validation binary_cross_entropy = 0.343425
Epoch 26
Validation binary_cross_entropy = 0.388390
Epoch 27
Validation binary_cross_entropy = 0.373439
Epoch 28
Validation binary_cross_entropy = 0.355052
Epoch 29
Loss = 2.7796e-01, PNorm = 89.5058, GNorm = 2.3041, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.388554
Epoch 30
Validation binary_cross_entropy = 0.458789
Epoch 31
Validation binary_cross_entropy = 0.409022
Epoch 32
Validation binary_cross_entropy = 0.407965
Epoch 33
Validation binary_cross_entropy = 0.445509
Epoch 34
Loss = 1.9010e-01, PNorm = 89.7416, GNorm = 1.1042, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.380044
Epoch 35
Validation binary_cross_entropy = 0.378019
Epoch 36
Validation binary_cross_entropy = 0.386808
Epoch 37
Validation binary_cross_entropy = 0.365507
Epoch 38
Validation binary_cross_entropy = 0.378514
Epoch 39
Loss = 1.9554e-01, PNorm = 89.9696, GNorm = 1.4517, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.380554
Epoch 40
Validation binary_cross_entropy = 0.404028
Epoch 41
Validation binary_cross_entropy = 0.427492
Epoch 42
Validation binary_cross_entropy = 0.400147
Epoch 43
Validation binary_cross_entropy = 0.401356
Epoch 44
Loss = 2.5165e-01, PNorm = 90.2331, GNorm = 2.5132, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.372511
Epoch 45
Validation binary_cross_entropy = 0.362695
Epoch 46
Validation binary_cross_entropy = 0.363759
Epoch 47
Validation binary_cross_entropy = 0.367877
Epoch 48
Validation binary_cross_entropy = 0.374480
Epoch 49
Loss = 1.6306e-01, PNorm = 90.5104, GNorm = 1.3340, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.396942
Epoch 50
Validation binary_cross_entropy = 0.431654
Epoch 51
Validation binary_cross_entropy = 0.429679
Epoch 52
Validation binary_cross_entropy = 0.426043
Epoch 53
Validation binary_cross_entropy = 0.433301
Epoch 54
Loss = 2.0239e-01, PNorm = 90.8258, GNorm = 2.2399, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.440084
Epoch 55
Validation binary_cross_entropy = 0.433539
Epoch 56
Validation binary_cross_entropy = 0.457423
Epoch 57
Validation binary_cross_entropy = 0.445943
Epoch 58
Validation binary_cross_entropy = 0.466558
Epoch 59
Loss = 1.1577e-01, PNorm = 91.1295, GNorm = 1.0448, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.501606
Epoch 60
Validation binary_cross_entropy = 0.522563
Epoch 61
Validation binary_cross_entropy = 0.507845
Epoch 62
Validation binary_cross_entropy = 0.486925
Epoch 63
Validation binary_cross_entropy = 0.457813
Epoch 64
Loss = 1.4202e-01, PNorm = 91.4422, GNorm = 1.5650, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.465129
Epoch 65
Validation binary_cross_entropy = 0.447266
Epoch 66
Validation binary_cross_entropy = 0.487989
Epoch 67
Validation binary_cross_entropy = 0.459921
Epoch 68
Validation binary_cross_entropy = 0.524281
Epoch 69
Loss = 1.9954e-01, PNorm = 91.7410, GNorm = 2.5147, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.464661
Epoch 70
Validation binary_cross_entropy = 0.479629
Epoch 71
Validation binary_cross_entropy = 0.471451
Epoch 72
Validation binary_cross_entropy = 0.474158
Epoch 73
Validation binary_cross_entropy = 0.489981
Epoch 74
Loss = 2.0747e-01, PNorm = 92.0380, GNorm = 1.9154, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.477577
Epoch 75
Validation binary_cross_entropy = 0.467416
Epoch 76
Validation binary_cross_entropy = 0.461047
Epoch 77
Validation binary_cross_entropy = 0.476377
Epoch 78
Validation binary_cross_entropy = 0.499574
Epoch 79
Loss = 1.2184e-01, PNorm = 92.3332, GNorm = 1.1509, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.488759
Epoch 80
Validation binary_cross_entropy = 0.489926
Epoch 81
Validation binary_cross_entropy = 0.483063
Epoch 82
Validation binary_cross_entropy = 0.478261
Epoch 83
Validation binary_cross_entropy = 0.491373
Epoch 84
Loss = 5.5305e-02, PNorm = 92.6198, GNorm = 0.8904, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.520486
Epoch 85
Validation binary_cross_entropy = 0.542436
Epoch 86
Validation binary_cross_entropy = 0.558337
Epoch 87
Validation binary_cross_entropy = 0.575463
Epoch 88
Validation binary_cross_entropy = 0.571314
Epoch 89
Loss = 5.7425e-02, PNorm = 92.8909, GNorm = 1.1210, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.593647
Epoch 90
Validation binary_cross_entropy = 0.641215
Epoch 91
Validation binary_cross_entropy = 0.629077
Epoch 92
Validation binary_cross_entropy = 0.619348
Epoch 93
Validation binary_cross_entropy = 0.650328
Epoch 94
Loss = 4.0251e-02, PNorm = 93.1634, GNorm = 0.7160, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.641903
Epoch 95
Validation binary_cross_entropy = 0.593359
Epoch 96
Validation binary_cross_entropy = 0.572915
Epoch 97
Validation binary_cross_entropy = 0.593771
Epoch 98
Validation binary_cross_entropy = 0.634189
Epoch 99
Loss = 6.4238e-02, PNorm = 93.4240, GNorm = 0.8941, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.615550
Epoch 100
Validation binary_cross_entropy = 0.602491
Epoch 101
Validation binary_cross_entropy = 0.597552
Epoch 102
Validation binary_cross_entropy = 0.605252
Epoch 103
Validation binary_cross_entropy = 0.640967
Epoch 104
Loss = 7.9594e-02, PNorm = 93.6546, GNorm = 1.8867, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.652432
Epoch 105
Validation binary_cross_entropy = 0.623862
Epoch 106
Validation binary_cross_entropy = 0.638624
Epoch 107
Validation binary_cross_entropy = 0.674691
Epoch 108
Validation binary_cross_entropy = 0.664138
Epoch 109
Loss = 4.1530e-02, PNorm = 93.8886, GNorm = 1.2339, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.713304
Epoch 110
Validation binary_cross_entropy = 0.656920
Epoch 111
Validation binary_cross_entropy = 0.671330
Epoch 112
Validation binary_cross_entropy = 0.671959
Epoch 113
Validation binary_cross_entropy = 0.651185
Epoch 114
Loss = 4.9158e-02, PNorm = 94.1432, GNorm = 0.9050, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.642306
Epoch 115
Validation binary_cross_entropy = 0.661499
Epoch 116
Validation binary_cross_entropy = 0.682200
Epoch 117
Validation binary_cross_entropy = 0.758096
Epoch 118
Validation binary_cross_entropy = 0.744684
Epoch 119
Loss = 4.7359e-02, PNorm = 94.3904, GNorm = 0.9060, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.675263
Epoch 120
Validation binary_cross_entropy = 0.668655
Epoch 121
Validation binary_cross_entropy = 0.712516
Epoch 122
Validation binary_cross_entropy = 0.727471
Epoch 123
Validation binary_cross_entropy = 0.695524
Epoch 124
Loss = 4.2370e-02, PNorm = 94.6272, GNorm = 0.8737, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.694481
Epoch 125
Validation binary_cross_entropy = 0.728068
Epoch 126
Validation binary_cross_entropy = 0.757709
Epoch 127
Validation binary_cross_entropy = 0.691299
Epoch 128
Validation binary_cross_entropy = 0.687612
Epoch 129
Loss = 2.7400e-02, PNorm = 94.8492, GNorm = 0.5299, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.700896
Epoch 130
Validation binary_cross_entropy = 0.736593
Epoch 131
Validation binary_cross_entropy = 0.756761
Epoch 132
Validation binary_cross_entropy = 0.777407
Epoch 133
Validation binary_cross_entropy = 0.803983
Epoch 134
Loss = 3.4289e-02, PNorm = 95.0917, GNorm = 0.3568, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.823918
Epoch 135
Validation binary_cross_entropy = 0.893265
Epoch 136
Validation binary_cross_entropy = 0.791759
Epoch 137
Validation binary_cross_entropy = 0.756254
Epoch 138
Validation binary_cross_entropy = 0.741612
Epoch 139
Loss = 2.8476e-02, PNorm = 95.3290, GNorm = 0.4479, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.704418
Epoch 140
Validation binary_cross_entropy = 0.696317
Epoch 141
Validation binary_cross_entropy = 0.733165
Epoch 142
Validation binary_cross_entropy = 0.784926
Epoch 143
Validation binary_cross_entropy = 0.827799
Epoch 144
Loss = 9.6983e-02, PNorm = 95.5853, GNorm = 1.7871, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.840832
Epoch 145
Validation binary_cross_entropy = 0.829167
Epoch 146
Validation binary_cross_entropy = 0.791570
Epoch 147
Validation binary_cross_entropy = 0.763018
Epoch 148
Validation binary_cross_entropy = 0.757477
Epoch 149
Loss = 3.7369e-02, PNorm = 95.8018, GNorm = 0.7310, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.774109
Epoch 150
Validation binary_cross_entropy = 0.778370
Epoch 151
Validation binary_cross_entropy = 0.792376
Epoch 152
Validation binary_cross_entropy = 0.832487
Epoch 153
Validation binary_cross_entropy = 0.862764
Epoch 154
Loss = 4.6096e-02, PNorm = 96.0079, GNorm = 0.9948, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.887028
Epoch 155
Validation binary_cross_entropy = 0.929397
Epoch 156
Validation binary_cross_entropy = 0.936100
Epoch 157
Validation binary_cross_entropy = 0.934004
Epoch 158
Validation binary_cross_entropy = 0.934416
Epoch 159
Loss = 3.4721e-02, PNorm = 96.1782, GNorm = 0.6440, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.925351
Epoch 160
Validation binary_cross_entropy = 0.900600
Epoch 161
Validation binary_cross_entropy = 0.910786
Epoch 162
Validation binary_cross_entropy = 0.874165
Epoch 163
Validation binary_cross_entropy = 0.842814
Epoch 164
Loss = 8.2370e-02, PNorm = 96.3402, GNorm = 1.4739, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.871704
Epoch 165
Validation binary_cross_entropy = 0.906738
Epoch 166
Validation binary_cross_entropy = 0.909813
Epoch 167
Validation binary_cross_entropy = 0.880454
Epoch 168
Validation binary_cross_entropy = 0.887118
Epoch 169
Loss = 3.2585e-02, PNorm = 96.5184, GNorm = 0.6940, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.877219
Epoch 170
Validation binary_cross_entropy = 0.895232
Epoch 171
Validation binary_cross_entropy = 0.915203
Epoch 172
Validation binary_cross_entropy = 0.916945
Epoch 173
Validation binary_cross_entropy = 0.924060
Epoch 174
Loss = 2.6503e-02, PNorm = 96.7001, GNorm = 0.4273, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.947245
Epoch 175
Validation binary_cross_entropy = 0.945404
Epoch 176
Validation binary_cross_entropy = 0.938748
Epoch 177
Validation binary_cross_entropy = 0.937096
Epoch 178
Validation binary_cross_entropy = 0.957185
Epoch 179
Loss = 1.6622e-02, PNorm = 96.8271, GNorm = 0.4554, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.963903
Epoch 180
Validation binary_cross_entropy = 0.955640
Epoch 181
Validation binary_cross_entropy = 0.952786
Epoch 182
Validation binary_cross_entropy = 0.952720
Epoch 183
Validation binary_cross_entropy = 0.967357
Epoch 184
Loss = 1.0555e-02, PNorm = 96.9688, GNorm = 0.0839, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.986569
Epoch 185
Validation binary_cross_entropy = 1.063663
Epoch 186
Validation binary_cross_entropy = 1.143945
Epoch 187
Validation binary_cross_entropy = 0.970121
Epoch 188
Validation binary_cross_entropy = 0.961530
Epoch 189
Loss = 6.1932e-02, PNorm = 97.1623, GNorm = 1.4173, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.976535
Epoch 190
Validation binary_cross_entropy = 0.957168
Epoch 191
Validation binary_cross_entropy = 1.031923
Epoch 192
Validation binary_cross_entropy = 1.037633
Epoch 193
Validation binary_cross_entropy = 1.048090
Epoch 194
Loss = 5.2608e-02, PNorm = 97.4110, GNorm = 1.3890, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.083963
Epoch 195
Validation binary_cross_entropy = 1.109407
Epoch 196
Validation binary_cross_entropy = 1.111944
Epoch 197
Validation binary_cross_entropy = 1.046015
Epoch 198
Validation binary_cross_entropy = 0.984217
Epoch 199
Loss = 6.5842e-03, PNorm = 97.5951, GNorm = 0.0238, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.965293
Epoch 200
Validation binary_cross_entropy = 0.955868
Epoch 201
Validation binary_cross_entropy = 0.996503
Epoch 202
Validation binary_cross_entropy = 0.974371
Epoch 203
Validation binary_cross_entropy = 0.977257
Epoch 204
Loss = 4.0463e-02, PNorm = 97.7433, GNorm = 1.5487, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.999675
Epoch 205
Validation binary_cross_entropy = 1.034621
Epoch 206
Validation binary_cross_entropy = 1.082193
Epoch 207
Validation binary_cross_entropy = 1.075798
Epoch 208
Validation binary_cross_entropy = 1.047945
Epoch 209
Loss = 9.0059e-03, PNorm = 97.9096, GNorm = 0.0900, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.031697
Epoch 210
Validation binary_cross_entropy = 1.022308
Epoch 211
Validation binary_cross_entropy = 1.029719
Epoch 212
Validation binary_cross_entropy = 1.049008
Epoch 213
Validation binary_cross_entropy = 1.037438
Epoch 214
Loss = 1.2980e-02, PNorm = 98.0348, GNorm = 0.5520, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.018695
Epoch 215
Validation binary_cross_entropy = 0.995657
Epoch 216
Validation binary_cross_entropy = 0.979270
Epoch 217
Validation binary_cross_entropy = 0.975509
Epoch 218
Validation binary_cross_entropy = 0.990403
Epoch 219
Loss = 2.6104e-02, PNorm = 98.1434, GNorm = 0.0994, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.032372
Epoch 220
Validation binary_cross_entropy = 1.063534
Epoch 221
Validation binary_cross_entropy = 1.062278
Epoch 222
Validation binary_cross_entropy = 1.054423
Epoch 223
Validation binary_cross_entropy = 1.047380
Epoch 224
Loss = 6.6865e-02, PNorm = 98.2503, GNorm = 0.0912, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.130286
Epoch 225
Validation binary_cross_entropy = 1.213488
Epoch 226
Validation binary_cross_entropy = 1.269670
Epoch 227
Validation binary_cross_entropy = 1.258242
Epoch 228
Validation binary_cross_entropy = 1.800797
Epoch 229
Loss = 3.2796e-02, PNorm = 98.3425, GNorm = 2.1565, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.216310
Epoch 230
Validation binary_cross_entropy = 1.160384
Epoch 231
Validation binary_cross_entropy = 1.107544
Epoch 232
Validation binary_cross_entropy = 1.043624
Epoch 233
Validation binary_cross_entropy = 1.001757
Epoch 234
Loss = 1.0894e-02, PNorm = 98.4358, GNorm = 0.0565, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.986352
Epoch 235
Validation binary_cross_entropy = 1.004658
Epoch 236
Validation binary_cross_entropy = 1.046843
Epoch 237
Validation binary_cross_entropy = 1.146826
Epoch 238
Validation binary_cross_entropy = 1.204762
Epoch 239
Loss = 1.2836e-02, PNorm = 98.5623, GNorm = 0.4392, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.195298
Epoch 240
Validation binary_cross_entropy = 1.188127
Epoch 241
Validation binary_cross_entropy = 1.197552
Epoch 242
Validation binary_cross_entropy = 1.210681
Epoch 243
Validation binary_cross_entropy = 1.206422
Epoch 244
Loss = 9.0266e-03, PNorm = 98.6806, GNorm = 0.5544, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.216301
Epoch 245
Validation binary_cross_entropy = 1.225540
Epoch 246
Validation binary_cross_entropy = 1.201233
Epoch 247
Validation binary_cross_entropy = 1.169197
Epoch 248
Validation binary_cross_entropy = 1.149724
Epoch 249
Loss = 9.4504e-03, PNorm = 98.8030, GNorm = 0.3913, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.147396
Epoch 250
Validation binary_cross_entropy = 1.171296
Epoch 251
Validation binary_cross_entropy = 1.130235
Epoch 252
Validation binary_cross_entropy = 1.087997
Epoch 253
Validation binary_cross_entropy = 1.093823
Epoch 254
Loss = 2.9307e-03, PNorm = 98.9021, GNorm = 0.1115, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.126113
Epoch 255
Validation binary_cross_entropy = 1.194261
Epoch 256
Validation binary_cross_entropy = 1.243761
Epoch 257
Validation binary_cross_entropy = 1.261212
Epoch 258
Validation binary_cross_entropy = 1.231630
Epoch 259
Loss = 1.7140e-02, PNorm = 99.0079, GNorm = 0.9600, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.226500
Epoch 260
Validation binary_cross_entropy = 1.254266
Epoch 261
Validation binary_cross_entropy = 1.314409
Epoch 262
Validation binary_cross_entropy = 1.257720
Epoch 263
Validation binary_cross_entropy = 1.127563
Epoch 264
Loss = 1.0231e-02, PNorm = 99.1285, GNorm = 0.2425, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.063100
Epoch 265
Validation binary_cross_entropy = 1.024058
Epoch 266
Validation binary_cross_entropy = 1.005360
Epoch 267
Validation binary_cross_entropy = 1.100531
Epoch 268
Validation binary_cross_entropy = 1.143912
Epoch 269
Loss = 4.6105e-02, PNorm = 99.3190, GNorm = 1.8211, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.024976
Epoch 270
Validation binary_cross_entropy = 0.943360
Epoch 271
Validation binary_cross_entropy = 0.922677
Epoch 272
Validation binary_cross_entropy = 0.910247
Epoch 273
Validation binary_cross_entropy = 0.919246
Epoch 274
Loss = 2.9708e-02, PNorm = 99.5109, GNorm = 1.0013, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.949390
Epoch 275
Validation binary_cross_entropy = 0.969476
Epoch 276
Validation binary_cross_entropy = 0.895292
Epoch 277
Validation binary_cross_entropy = 0.865473
Epoch 278
Validation binary_cross_entropy = 0.854640
Epoch 279
Loss = 2.2691e-02, PNorm = 99.6757, GNorm = 0.1561, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.854906
Epoch 280
Validation binary_cross_entropy = 0.874165
Epoch 281
Validation binary_cross_entropy = 0.897599
Epoch 282
Validation binary_cross_entropy = 0.915343
Epoch 283
Validation binary_cross_entropy = 0.935528
Epoch 284
Loss = 2.2214e-02, PNorm = 99.8461, GNorm = 0.8405, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.962794
Epoch 285
Validation binary_cross_entropy = 0.976576
Epoch 286
Validation binary_cross_entropy = 0.997398
Epoch 287
Validation binary_cross_entropy = 1.016336
Epoch 288
Validation binary_cross_entropy = 1.052675
Epoch 289
Loss = 6.1705e-03, PNorm = 99.9787, GNorm = 0.1368, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.099498
Epoch 290
Validation binary_cross_entropy = 1.115611
Epoch 291
Validation binary_cross_entropy = 1.116791
Epoch 292
Validation binary_cross_entropy = 1.117435
Epoch 293
Validation binary_cross_entropy = 1.120405
Epoch 294
Loss = 1.1831e-02, PNorm = 100.0755, GNorm = 0.8720, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.139825
Epoch 295
Validation binary_cross_entropy = 1.183715
Epoch 296
Validation binary_cross_entropy = 1.219716
Epoch 297
Validation binary_cross_entropy = 1.229129
Epoch 298
Validation binary_cross_entropy = 1.214608
Epoch 299
Loss = 1.9993e-02, PNorm = 100.1520, GNorm = 0.6568, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.217342
Model 0 best validation binary_cross_entropy = 0.310109 on epoch 14
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.193772
Ensemble test binary_cross_entropy = 0.193772
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.422135
Epoch 1
Validation binary_cross_entropy = 0.518092
Epoch 2
Validation binary_cross_entropy = 0.744488
Epoch 3
Validation binary_cross_entropy = 0.286152
Epoch 4
Loss = 6.2248e-01, PNorm = 87.7940, GNorm = 6.6829, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.437166
Epoch 5
Validation binary_cross_entropy = 0.741987
Epoch 6
Validation binary_cross_entropy = 0.300930
Epoch 7
Validation binary_cross_entropy = 0.308058
Epoch 8
Validation binary_cross_entropy = 0.469052
Epoch 9
Loss = 4.6767e-01, PNorm = 88.2047, GNorm = 2.8961, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.523658
Epoch 10
Validation binary_cross_entropy = 0.370307
Epoch 11
Validation binary_cross_entropy = 0.310720
Epoch 12
Validation binary_cross_entropy = 0.355089
Epoch 13
Validation binary_cross_entropy = 0.412749
Epoch 14
Loss = 2.8024e-01, PNorm = 88.6054, GNorm = 1.7883, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.370147
Epoch 15
Validation binary_cross_entropy = 0.361480
Epoch 16
Validation binary_cross_entropy = 0.360320
Epoch 17
Validation binary_cross_entropy = 0.347891
Epoch 18
Validation binary_cross_entropy = 0.350265
Epoch 19
Loss = 2.5661e-01, PNorm = 88.8820, GNorm = 1.2010, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.440863
Epoch 20
Validation binary_cross_entropy = 0.357586
Epoch 21
Validation binary_cross_entropy = 0.307259
Epoch 22
Validation binary_cross_entropy = 0.319587
Epoch 23
Validation binary_cross_entropy = 0.419201
Epoch 24
Loss = 2.4030e-01, PNorm = 89.0996, GNorm = 1.3845, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.336185
Epoch 25
Validation binary_cross_entropy = 0.315583
Epoch 26
Validation binary_cross_entropy = 0.358719
Epoch 27
Validation binary_cross_entropy = 0.432865
Epoch 28
Validation binary_cross_entropy = 0.369435
Epoch 29
Loss = 3.9285e-01, PNorm = 89.3067, GNorm = 2.2878, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.319775
Epoch 30
Validation binary_cross_entropy = 0.317619
Epoch 31
Validation binary_cross_entropy = 0.393331
Epoch 32
Validation binary_cross_entropy = 0.361174
Epoch 33
Validation binary_cross_entropy = 0.327728
Epoch 34
Loss = 1.9676e-01, PNorm = 89.5209, GNorm = 2.0033, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.335503
Epoch 35
Validation binary_cross_entropy = 0.368003
Epoch 36
Validation binary_cross_entropy = 0.393195
Epoch 37
Validation binary_cross_entropy = 0.353333
Epoch 38
Validation binary_cross_entropy = 0.340855
Epoch 39
Loss = 2.6909e-01, PNorm = 89.7581, GNorm = 1.5624, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.369448
Epoch 40
Validation binary_cross_entropy = 0.385602
Epoch 41
Validation binary_cross_entropy = 0.346190
Epoch 42
Validation binary_cross_entropy = 0.344380
Epoch 43
Validation binary_cross_entropy = 0.338047
Epoch 44
Loss = 2.3353e-01, PNorm = 89.9842, GNorm = 1.4972, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.341144
Epoch 45
Validation binary_cross_entropy = 0.363476
Epoch 46
Validation binary_cross_entropy = 0.348165
Epoch 47
Validation binary_cross_entropy = 0.355577
Epoch 48
Validation binary_cross_entropy = 0.384186
Epoch 49
Loss = 2.0196e-01, PNorm = 90.2408, GNorm = 1.2820, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.384625
Epoch 50
Validation binary_cross_entropy = 0.393981
Epoch 51
Validation binary_cross_entropy = 0.407628
Epoch 52
Validation binary_cross_entropy = 0.417657
Epoch 53
Validation binary_cross_entropy = 0.415478
Epoch 54
Loss = 1.4653e-01, PNorm = 90.5199, GNorm = 1.0742, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.416755
Epoch 55
Validation binary_cross_entropy = 0.412447
Epoch 56
Validation binary_cross_entropy = 0.409288
Epoch 57
Validation binary_cross_entropy = 0.416868
Epoch 58
Validation binary_cross_entropy = 0.421163
Epoch 59
Loss = 1.6116e-01, PNorm = 90.8147, GNorm = 1.3159, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.396516
Epoch 60
Validation binary_cross_entropy = 0.390026
Epoch 61
Validation binary_cross_entropy = 0.382288
Epoch 62
Validation binary_cross_entropy = 0.394852
Epoch 63
Validation binary_cross_entropy = 0.394672
Epoch 64
Loss = 1.1163e-01, PNorm = 91.1042, GNorm = 0.7212, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.434726
Epoch 65
Validation binary_cross_entropy = 0.446013
Epoch 66
Validation binary_cross_entropy = 0.417156
Epoch 67
Validation binary_cross_entropy = 0.441471
Epoch 68
Validation binary_cross_entropy = 0.435203
Epoch 69
Loss = 8.9401e-02, PNorm = 91.4395, GNorm = 1.1622, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.455013
Epoch 70
Validation binary_cross_entropy = 0.455105
Epoch 71
Validation binary_cross_entropy = 0.477074
Epoch 72
Validation binary_cross_entropy = 0.468553
Epoch 73
Validation binary_cross_entropy = 0.482387
Epoch 74
Loss = 6.7900e-02, PNorm = 91.7470, GNorm = 0.3851, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.518468
Epoch 75
Validation binary_cross_entropy = 0.452606
Epoch 76
Validation binary_cross_entropy = 0.478773
Epoch 77
Validation binary_cross_entropy = 0.441076
Epoch 78
Validation binary_cross_entropy = 0.417357
Epoch 79
Loss = 1.9399e-01, PNorm = 92.0326, GNorm = 2.3828, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.410829
Epoch 80
Validation binary_cross_entropy = 0.407577
Epoch 81
Validation binary_cross_entropy = 0.418852
Epoch 82
Validation binary_cross_entropy = 0.471669
Epoch 83
Validation binary_cross_entropy = 0.449363
Epoch 84
Loss = 9.9905e-02, PNorm = 92.3518, GNorm = 0.8511, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.449539
Epoch 85
Validation binary_cross_entropy = 0.464188
Epoch 86
Validation binary_cross_entropy = 0.474262
Epoch 87
Validation binary_cross_entropy = 0.488680
Epoch 88
Validation binary_cross_entropy = 0.507602
Epoch 89
Loss = 1.0868e-01, PNorm = 92.6683, GNorm = 1.4760, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.524235
Epoch 90
Validation binary_cross_entropy = 0.533384
Epoch 91
Validation binary_cross_entropy = 0.516025
Epoch 92
Validation binary_cross_entropy = 0.498847
Epoch 93
Validation binary_cross_entropy = 0.478541
Epoch 94
Loss = 8.1955e-02, PNorm = 92.9231, GNorm = 0.7657, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.479980
Epoch 95
Validation binary_cross_entropy = 0.498801
Epoch 96
Validation binary_cross_entropy = 0.502735
Epoch 97
Validation binary_cross_entropy = 0.505612
Epoch 98
Validation binary_cross_entropy = 0.528158
Epoch 99
Loss = 7.2559e-02, PNorm = 93.1630, GNorm = 1.4270, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.555125
Epoch 100
Validation binary_cross_entropy = 0.596494
Epoch 101
Validation binary_cross_entropy = 0.596606
Epoch 102
Validation binary_cross_entropy = 0.570271
Epoch 103
Validation binary_cross_entropy = 0.552311
Epoch 104
Loss = 1.7195e-01, PNorm = 93.3830, GNorm = 2.2042, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.567284
Epoch 105
Validation binary_cross_entropy = 0.574126
Epoch 106
Validation binary_cross_entropy = 0.535263
Epoch 107
Validation binary_cross_entropy = 0.509667
Epoch 108
Validation binary_cross_entropy = 0.524198
Epoch 109
Loss = 8.6436e-02, PNorm = 93.6862, GNorm = 1.4597, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.549294
Epoch 110
Validation binary_cross_entropy = 0.576550
Epoch 111
Validation binary_cross_entropy = 0.556646
Epoch 112
Validation binary_cross_entropy = 0.525491
Epoch 113
Validation binary_cross_entropy = 0.540368
Epoch 114
Loss = 8.7971e-02, PNorm = 94.0001, GNorm = 1.0939, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.573696
Epoch 115
Validation binary_cross_entropy = 0.599640
Epoch 116
Validation binary_cross_entropy = 0.609651
Epoch 117
Validation binary_cross_entropy = 0.604525
Epoch 118
Validation binary_cross_entropy = 0.624568
Epoch 119
Loss = 6.9564e-02, PNorm = 94.3021, GNorm = 0.7839, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.647737
Epoch 120
Validation binary_cross_entropy = 0.608863
Epoch 121
Validation binary_cross_entropy = 0.582731
Epoch 122
Validation binary_cross_entropy = 0.565614
Epoch 123
Validation binary_cross_entropy = 0.563297
Epoch 124
Loss = 6.3603e-02, PNorm = 94.6493, GNorm = 0.9016, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.579438
Epoch 125
Validation binary_cross_entropy = 0.577540
Epoch 126
Validation binary_cross_entropy = 0.570564
Epoch 127
Validation binary_cross_entropy = 0.602757
Epoch 128
Validation binary_cross_entropy = 0.684532
Epoch 129
Loss = 4.8886e-02, PNorm = 94.9888, GNorm = 0.9341, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.650795
Epoch 130
Validation binary_cross_entropy = 0.647344
Epoch 131
Validation binary_cross_entropy = 0.669923
Epoch 132
Validation binary_cross_entropy = 0.582916
Epoch 133
Validation binary_cross_entropy = 0.559927
Epoch 134
Loss = 3.3167e-02, PNorm = 95.2913, GNorm = 1.2201, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.576858
Epoch 135
Validation binary_cross_entropy = 0.544588
Epoch 136
Validation binary_cross_entropy = 0.559826
Epoch 137
Validation binary_cross_entropy = 0.570009
Epoch 138
Validation binary_cross_entropy = 0.569594
Epoch 139
Loss = 8.8865e-02, PNorm = 95.6016, GNorm = 0.8082, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.582082
Epoch 140
Validation binary_cross_entropy = 0.596535
Epoch 141
Validation binary_cross_entropy = 0.612365
Epoch 142
Validation binary_cross_entropy = 0.615576
Epoch 143
Validation binary_cross_entropy = 0.626031
Epoch 144
Loss = 4.0432e-02, PNorm = 95.8687, GNorm = 0.4905, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.638913
Epoch 145
Validation binary_cross_entropy = 0.656493
Epoch 146
Validation binary_cross_entropy = 0.692128
Epoch 147
Validation binary_cross_entropy = 0.722581
Epoch 148
Validation binary_cross_entropy = 0.739148
Epoch 149
Loss = 6.2165e-02, PNorm = 96.0857, GNorm = 0.5578, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.750121
Epoch 150
Validation binary_cross_entropy = 0.749498
Epoch 151
Validation binary_cross_entropy = 0.752181
Epoch 152
Validation binary_cross_entropy = 0.759307
Epoch 153
Validation binary_cross_entropy = 0.744432
Epoch 154
Loss = 3.0633e-02, PNorm = 96.2741, GNorm = 0.3350, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.747479
Epoch 155
Validation binary_cross_entropy = 0.747961
Epoch 156
Validation binary_cross_entropy = 0.756007
Epoch 157
Validation binary_cross_entropy = 0.753456
Epoch 158
Validation binary_cross_entropy = 0.759674
Epoch 159
Loss = 1.3361e-02, PNorm = 96.4312, GNorm = 0.2473, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.765741
Epoch 160
Validation binary_cross_entropy = 0.787251
Epoch 161
Validation binary_cross_entropy = 0.786496
Epoch 162
Validation binary_cross_entropy = 0.777852
Epoch 163
Validation binary_cross_entropy = 0.763171
Epoch 164
Loss = 3.7951e-02, PNorm = 96.5905, GNorm = 0.4580, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.746169
Epoch 165
Validation binary_cross_entropy = 0.735629
Epoch 166
Validation binary_cross_entropy = 0.729341
Epoch 167
Validation binary_cross_entropy = 0.758280
Epoch 168
Validation binary_cross_entropy = 0.783293
Epoch 169
Loss = 3.9787e-02, PNorm = 96.7650, GNorm = 0.4563, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.761305
Epoch 170
Validation binary_cross_entropy = 0.768171
Epoch 171
Validation binary_cross_entropy = 0.793920
Epoch 172
Validation binary_cross_entropy = 0.848164
Epoch 173
Validation binary_cross_entropy = 0.879739
Epoch 174
Loss = 6.7256e-02, PNorm = 96.9421, GNorm = 0.6191, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.830208
Epoch 175
Validation binary_cross_entropy = 0.783366
Epoch 176
Validation binary_cross_entropy = 0.775658
Epoch 177
Validation binary_cross_entropy = 0.808616
Epoch 178
Validation binary_cross_entropy = 0.849912
Epoch 179
Loss = 3.0564e-02, PNorm = 97.1575, GNorm = 0.7113, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.880386
Epoch 180
Validation binary_cross_entropy = 0.864444
Epoch 181
Validation binary_cross_entropy = 0.864567
Epoch 182
Validation binary_cross_entropy = 0.884545
Epoch 183
Validation binary_cross_entropy = 0.859241
Epoch 184
Loss = 1.6326e-01, PNorm = 97.3932, GNorm = 2.2486, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.848347
Epoch 185
Validation binary_cross_entropy = 0.844935
Epoch 186
Validation binary_cross_entropy = 0.804439
Epoch 187
Validation binary_cross_entropy = 0.783493
Epoch 188
Validation binary_cross_entropy = 0.749084
Epoch 189
Loss = 6.0860e-02, PNorm = 97.6524, GNorm = 1.2567, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.750834
Epoch 190
Validation binary_cross_entropy = 0.762597
Epoch 191
Validation binary_cross_entropy = 0.772835
Epoch 192
Validation binary_cross_entropy = 0.781532
Epoch 193
Validation binary_cross_entropy = 0.814154
Epoch 194
Loss = 2.2482e-02, PNorm = 97.8657, GNorm = 0.5164, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.856156
Epoch 195
Validation binary_cross_entropy = 0.869948
Epoch 196
Validation binary_cross_entropy = 0.853038
Epoch 197
Validation binary_cross_entropy = 0.835314
Epoch 198
Validation binary_cross_entropy = 0.836605
Epoch 199
Loss = 2.5917e-02, PNorm = 98.0338, GNorm = 1.0013, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.858158
Epoch 200
Validation binary_cross_entropy = 0.890770
Epoch 201
Validation binary_cross_entropy = 0.922118
Epoch 202
Validation binary_cross_entropy = 0.930719
Epoch 203
Validation binary_cross_entropy = 0.929584
Epoch 204
Loss = 1.1149e-02, PNorm = 98.1483, GNorm = 0.3149, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.921172
Epoch 205
Validation binary_cross_entropy = 0.906863
Epoch 206
Validation binary_cross_entropy = 0.906090
Epoch 207
Validation binary_cross_entropy = 0.915436
Epoch 208
Validation binary_cross_entropy = 0.924784
Epoch 209
Loss = 2.9143e-02, PNorm = 98.2690, GNorm = 0.7587, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.935592
Epoch 210
Validation binary_cross_entropy = 0.942875
Epoch 211
Validation binary_cross_entropy = 0.937170
Epoch 212
Validation binary_cross_entropy = 0.918284
Epoch 213
Validation binary_cross_entropy = 0.901355
Epoch 214
Loss = 1.5773e-02, PNorm = 98.3953, GNorm = 0.1393, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.921210
Epoch 215
Validation binary_cross_entropy = 0.952492
Epoch 216
Validation binary_cross_entropy = 0.995235
Epoch 217
Validation binary_cross_entropy = 0.976400
Epoch 218
Validation binary_cross_entropy = 0.924596
Epoch 219
Loss = 6.3929e-03, PNorm = 98.5163, GNorm = 0.1951, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.879837
Epoch 220
Validation binary_cross_entropy = 0.894177
Epoch 221
Validation binary_cross_entropy = 0.940454
Epoch 222
Validation binary_cross_entropy = 0.986028
Epoch 223
Validation binary_cross_entropy = 1.015130
Epoch 224
Loss = 4.0392e-02, PNorm = 98.6445, GNorm = 1.1843, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.000644
Epoch 225
Validation binary_cross_entropy = 0.986615
Epoch 226
Validation binary_cross_entropy = 1.011831
Epoch 227
Validation binary_cross_entropy = 1.047787
Epoch 228
Validation binary_cross_entropy = 1.096686
Epoch 229
Loss = 2.4138e-02, PNorm = 98.7718, GNorm = 0.5521, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.110218
Epoch 230
Validation binary_cross_entropy = 1.091219
Epoch 231
Validation binary_cross_entropy = 1.053706
Epoch 232
Validation binary_cross_entropy = 1.022579
Epoch 233
Validation binary_cross_entropy = 1.013305
Epoch 234
Loss = 1.3399e-02, PNorm = 98.9096, GNorm = 0.5994, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.014622
Epoch 235
Validation binary_cross_entropy = 0.988284
Epoch 236
Validation binary_cross_entropy = 0.941160
Epoch 237
Validation binary_cross_entropy = 0.932634
Epoch 238
Validation binary_cross_entropy = 0.957194
Epoch 239
Loss = 2.1777e-02, PNorm = 99.0698, GNorm = 0.4792, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.015625
Epoch 240
Validation binary_cross_entropy = 0.983539
Epoch 241
Validation binary_cross_entropy = 0.910855
Epoch 242
Validation binary_cross_entropy = 0.830149
Epoch 243
Validation binary_cross_entropy = 0.813962
Epoch 244
Loss = 1.6012e-02, PNorm = 99.2593, GNorm = 0.5104, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.833732
Epoch 245
Validation binary_cross_entropy = 0.861702
Epoch 246
Validation binary_cross_entropy = 0.883929
Epoch 247
Validation binary_cross_entropy = 0.886867
Epoch 248
Validation binary_cross_entropy = 0.918197
Epoch 249
Loss = 3.2625e-02, PNorm = 99.4177, GNorm = 0.7774, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.957174
Epoch 250
Validation binary_cross_entropy = 0.966112
Epoch 251
Validation binary_cross_entropy = 0.976583
Epoch 252
Validation binary_cross_entropy = 0.992676
Epoch 253
Validation binary_cross_entropy = 1.010235
Epoch 254
Loss = 1.8885e-02, PNorm = 99.5647, GNorm = 0.9256, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.974911
Epoch 255
Validation binary_cross_entropy = 0.929912
Epoch 256
Validation binary_cross_entropy = 0.908301
Epoch 257
Validation binary_cross_entropy = 0.895317
Epoch 258
Validation binary_cross_entropy = 0.911270
Epoch 259
Loss = 4.3994e-02, PNorm = 99.6791, GNorm = 0.1054, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.986977
Epoch 260
Validation binary_cross_entropy = 1.067695
Epoch 261
Validation binary_cross_entropy = 1.078404
Epoch 262
Validation binary_cross_entropy = 0.995177
Epoch 263
Validation binary_cross_entropy = 0.916031
Epoch 264
Loss = 8.0061e-03, PNorm = 99.8043, GNorm = 0.0252, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.881891
Epoch 265
Validation binary_cross_entropy = 0.862954
Epoch 266
Validation binary_cross_entropy = 0.858793
Epoch 267
Validation binary_cross_entropy = 0.867049
Epoch 268
Validation binary_cross_entropy = 0.889041
Epoch 269
Loss = 1.5730e-02, PNorm = 99.9099, GNorm = 0.5221, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.943036
Epoch 270
Validation binary_cross_entropy = 0.999000
Epoch 271
Validation binary_cross_entropy = 1.033541
Epoch 272
Validation binary_cross_entropy = 1.026409
Epoch 273
Validation binary_cross_entropy = 0.985955
Epoch 274
Loss = 2.3454e-03, PNorm = 100.0095, GNorm = 0.0846, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.958835
Epoch 275
Validation binary_cross_entropy = 0.942712
Epoch 276
Validation binary_cross_entropy = 0.941638
Epoch 277
Validation binary_cross_entropy = 0.960598
Epoch 278
Validation binary_cross_entropy = 0.992426
Epoch 279
Loss = 7.3835e-03, PNorm = 100.0944, GNorm = 0.0773, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.043224
Epoch 280
Validation binary_cross_entropy = 1.072508
Epoch 281
Validation binary_cross_entropy = 1.073311
Epoch 282
Validation binary_cross_entropy = 1.060045
Epoch 283
Validation binary_cross_entropy = 1.039185
Epoch 284
Loss = 2.8438e-03, PNorm = 100.2018, GNorm = 0.0773, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.023252
Epoch 285
Validation binary_cross_entropy = 1.026816
Epoch 286
Validation binary_cross_entropy = 1.033517
Epoch 287
Validation binary_cross_entropy = 1.039288
Epoch 288
Validation binary_cross_entropy = 1.054013
Epoch 289
Loss = 2.4299e-03, PNorm = 100.2907, GNorm = 0.0194, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.079969
Epoch 290
Validation binary_cross_entropy = 1.101834
Epoch 291
Validation binary_cross_entropy = 1.119583
Epoch 292
Validation binary_cross_entropy = 1.126093
Epoch 293
Validation binary_cross_entropy = 1.118567
Epoch 294
Loss = 1.7145e-02, PNorm = 100.3656, GNorm = 0.0433, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.142123
Epoch 295
Validation binary_cross_entropy = 1.162405
Epoch 296
Validation binary_cross_entropy = 1.167530
Epoch 297
Validation binary_cross_entropy = 1.161987
Epoch 298
Validation binary_cross_entropy = 1.162350
Epoch 299
Loss = 7.8132e-03, PNorm = 100.4364, GNorm = 0.3708, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.165593
Model 0 best validation binary_cross_entropy = 0.286152 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.218211
Ensemble test binary_cross_entropy = 0.218211
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.700963
Epoch 1
Validation binary_cross_entropy = 0.314051
Epoch 2
Validation binary_cross_entropy = 0.443100
Epoch 3
Validation binary_cross_entropy = 0.361059
Epoch 4
Loss = 5.4890e-01, PNorm = 87.8133, GNorm = 2.7543, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.288000
Epoch 5
Validation binary_cross_entropy = 0.326192
Epoch 6
Validation binary_cross_entropy = 0.297186
Epoch 7
Validation binary_cross_entropy = 0.598820
Epoch 8
Validation binary_cross_entropy = 0.494017
Epoch 9
Loss = 4.5263e-01, PNorm = 88.2502, GNorm = 2.0713, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.344040
Epoch 10
Validation binary_cross_entropy = 0.335532
Epoch 11
Validation binary_cross_entropy = 0.384247
Epoch 12
Validation binary_cross_entropy = 0.442487
Epoch 13
Validation binary_cross_entropy = 0.393799
Epoch 14
Loss = 3.7380e-01, PNorm = 88.7221, GNorm = 1.6872, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.362297
Epoch 15
Validation binary_cross_entropy = 0.329301
Epoch 16
Validation binary_cross_entropy = 0.341670
Epoch 17
Validation binary_cross_entropy = 0.399568
Epoch 18
Validation binary_cross_entropy = 0.457518
Epoch 19
Loss = 3.2521e-01, PNorm = 89.0333, GNorm = 1.9867, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.355136
Epoch 20
Validation binary_cross_entropy = 0.311632
Epoch 21
Validation binary_cross_entropy = 0.304050
Epoch 22
Validation binary_cross_entropy = 0.377016
Epoch 23
Validation binary_cross_entropy = 0.401753
Epoch 24
Loss = 3.2695e-01, PNorm = 89.2572, GNorm = 1.5029, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.330714
Epoch 25
Validation binary_cross_entropy = 0.329397
Epoch 26
Validation binary_cross_entropy = 0.389144
Epoch 27
Validation binary_cross_entropy = 0.464810
Epoch 28
Validation binary_cross_entropy = 0.420378
Epoch 29
Loss = 2.3150e-01, PNorm = 89.4540, GNorm = 1.4474, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.352418
Epoch 30
Validation binary_cross_entropy = 0.348817
Epoch 31
Validation binary_cross_entropy = 0.401198
Epoch 32
Validation binary_cross_entropy = 0.432306
Epoch 33
Validation binary_cross_entropy = 0.414072
Epoch 34
Loss = 2.0936e-01, PNorm = 89.6678, GNorm = 1.6020, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.381858
Epoch 35
Validation binary_cross_entropy = 0.382229
Epoch 36
Validation binary_cross_entropy = 0.394747
Epoch 37
Validation binary_cross_entropy = 0.387826
Epoch 38
Validation binary_cross_entropy = 0.379110
Epoch 39
Loss = 2.5203e-01, PNorm = 89.8920, GNorm = 1.9719, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.418834
Epoch 40
Validation binary_cross_entropy = 0.376333
Epoch 41
Validation binary_cross_entropy = 0.379193
Epoch 42
Validation binary_cross_entropy = 0.415569
Epoch 43
Validation binary_cross_entropy = 0.407700
Epoch 44
Loss = 1.8828e-01, PNorm = 90.1270, GNorm = 1.1491, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.378808
Epoch 45
Validation binary_cross_entropy = 0.384074
Epoch 46
Validation binary_cross_entropy = 0.421787
Epoch 47
Validation binary_cross_entropy = 0.445270
Epoch 48
Validation binary_cross_entropy = 0.442110
Epoch 49
Loss = 2.6211e-01, PNorm = 90.4058, GNorm = 1.7417, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.437678
Epoch 50
Validation binary_cross_entropy = 0.424756
Epoch 51
Validation binary_cross_entropy = 0.400420
Epoch 52
Validation binary_cross_entropy = 0.389507
Epoch 53
Validation binary_cross_entropy = 0.452881
Epoch 54
Loss = 1.7445e-01, PNorm = 90.6532, GNorm = 2.5719, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.515516
Epoch 55
Validation binary_cross_entropy = 0.392407
Epoch 56
Validation binary_cross_entropy = 0.399351
Epoch 57
Validation binary_cross_entropy = 0.402157
Epoch 58
Validation binary_cross_entropy = 0.417777
Epoch 59
Loss = 1.8371e-01, PNorm = 90.9551, GNorm = 1.5868, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.426556
Epoch 60
Validation binary_cross_entropy = 0.425980
Epoch 61
Validation binary_cross_entropy = 0.430169
Epoch 62
Validation binary_cross_entropy = 0.434028
Epoch 63
Validation binary_cross_entropy = 0.428957
Epoch 64
Loss = 1.3226e-01, PNorm = 91.1833, GNorm = 1.8456, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.439099
Epoch 65
Validation binary_cross_entropy = 0.432679
Epoch 66
Validation binary_cross_entropy = 0.445721
Epoch 67
Validation binary_cross_entropy = 0.429362
Epoch 68
Validation binary_cross_entropy = 0.461671
Epoch 69
Loss = 1.8790e-01, PNorm = 91.3905, GNorm = 0.9263, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.476414
Epoch 70
Validation binary_cross_entropy = 0.438281
Epoch 71
Validation binary_cross_entropy = 0.426384
Epoch 72
Validation binary_cross_entropy = 0.431216
Epoch 73
Validation binary_cross_entropy = 0.429817
Epoch 74
Loss = 7.7196e-02, PNorm = 91.6476, GNorm = 0.9365, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.438435
Epoch 75
Validation binary_cross_entropy = 0.444071
Epoch 76
Validation binary_cross_entropy = 0.450555
Epoch 77
Validation binary_cross_entropy = 0.470296
Epoch 78
Validation binary_cross_entropy = 0.479250
Epoch 79
Loss = 7.3407e-02, PNorm = 91.8963, GNorm = 0.7329, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.500032
Epoch 80
Validation binary_cross_entropy = 0.524381
Epoch 81
Validation binary_cross_entropy = 0.527578
Epoch 82
Validation binary_cross_entropy = 0.532908
Epoch 83
Validation binary_cross_entropy = 0.505734
Epoch 84
Loss = 5.2647e-02, PNorm = 92.1519, GNorm = 0.7085, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.504476
Epoch 85
Validation binary_cross_entropy = 0.504604
Epoch 86
Validation binary_cross_entropy = 0.519232
Epoch 87
Validation binary_cross_entropy = 0.536880
Epoch 88
Validation binary_cross_entropy = 0.527233
Epoch 89
Loss = 2.3015e-01, PNorm = 92.3974, GNorm = 2.0880, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.498033
Epoch 90
Validation binary_cross_entropy = 0.483494
Epoch 91
Validation binary_cross_entropy = 0.482059
Epoch 92
Validation binary_cross_entropy = 0.494187
Epoch 93
Validation binary_cross_entropy = 0.511824
Epoch 94
Loss = 6.8514e-02, PNorm = 92.6785, GNorm = 0.7476, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.529690
Epoch 95
Validation binary_cross_entropy = 0.562820
Epoch 96
Validation binary_cross_entropy = 0.617060
Epoch 97
Validation binary_cross_entropy = 0.637532
Epoch 98
Validation binary_cross_entropy = 0.650066
Epoch 99
Loss = 6.7807e-02, PNorm = 92.9995, GNorm = 0.6472, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.643261
Epoch 100
Validation binary_cross_entropy = 0.661497
Epoch 101
Validation binary_cross_entropy = 0.655074
Epoch 102
Validation binary_cross_entropy = 0.653791
Epoch 103
Validation binary_cross_entropy = 0.573487
Epoch 104
Loss = 1.5383e-01, PNorm = 93.2825, GNorm = 1.4769, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.542203
Epoch 105
Validation binary_cross_entropy = 0.534668
Epoch 106
Validation binary_cross_entropy = 0.537434
Epoch 107
Validation binary_cross_entropy = 0.524627
Epoch 108
Validation binary_cross_entropy = 0.523509
Epoch 109
Loss = 7.6426e-02, PNorm = 93.6005, GNorm = 0.7283, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.519209
Epoch 110
Validation binary_cross_entropy = 0.518993
Epoch 111
Validation binary_cross_entropy = 0.544656
Epoch 112
Validation binary_cross_entropy = 0.590845
Epoch 113
Validation binary_cross_entropy = 0.570172
Epoch 114
Loss = 7.6115e-02, PNorm = 93.9384, GNorm = 0.9630, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.581146
Epoch 115
Validation binary_cross_entropy = 0.594948
Epoch 116
Validation binary_cross_entropy = 0.600584
Epoch 117
Validation binary_cross_entropy = 0.587680
Epoch 118
Validation binary_cross_entropy = 0.577973
Epoch 119
Loss = 5.2593e-02, PNorm = 94.2468, GNorm = 0.4629, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.598865
Epoch 120
Validation binary_cross_entropy = 0.653038
Epoch 121
Validation binary_cross_entropy = 0.666724
Epoch 122
Validation binary_cross_entropy = 0.650297
Epoch 123
Validation binary_cross_entropy = 0.655509
Epoch 124
Loss = 1.0037e-01, PNorm = 94.4976, GNorm = 0.8122, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.726846
Epoch 125
Validation binary_cross_entropy = 0.702868
Epoch 126
Validation binary_cross_entropy = 0.678762
Epoch 127
Validation binary_cross_entropy = 0.692876
Epoch 128
Validation binary_cross_entropy = 0.689845
Epoch 129
Loss = 8.0265e-02, PNorm = 94.7766, GNorm = 1.3779, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.657607
Epoch 130
Validation binary_cross_entropy = 0.603871
Epoch 131
Validation binary_cross_entropy = 0.602502
Epoch 132
Validation binary_cross_entropy = 0.622010
Epoch 133
Validation binary_cross_entropy = 0.666483
Epoch 134
Loss = 1.3992e-01, PNorm = 95.0826, GNorm = 1.0207, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.711309
Epoch 135
Validation binary_cross_entropy = 0.713015
Epoch 136
Validation binary_cross_entropy = 0.724055
Epoch 137
Validation binary_cross_entropy = 0.724952
Epoch 138
Validation binary_cross_entropy = 0.721175
Epoch 139
Loss = 6.8407e-02, PNorm = 95.3480, GNorm = 0.1380, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.735416
Epoch 140
Validation binary_cross_entropy = 0.771422
Epoch 141
Validation binary_cross_entropy = 0.811848
Epoch 142
Validation binary_cross_entropy = 0.723345
Epoch 143
Validation binary_cross_entropy = 0.677575
Epoch 144
Loss = 5.0078e-02, PNorm = 95.5352, GNorm = 1.2651, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.663551
Epoch 145
Validation binary_cross_entropy = 0.664086
Epoch 146
Validation binary_cross_entropy = 0.677656
Epoch 147
Validation binary_cross_entropy = 0.682517
Epoch 148
Validation binary_cross_entropy = 0.697367
Epoch 149
Loss = 4.0419e-02, PNorm = 95.7459, GNorm = 0.9201, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.715484
Epoch 150
Validation binary_cross_entropy = 0.740580
Epoch 151
Validation binary_cross_entropy = 0.765724
Epoch 152
Validation binary_cross_entropy = 0.767868
Epoch 153
Validation binary_cross_entropy = 0.771920
Epoch 154
Loss = 3.5318e-02, PNorm = 95.9456, GNorm = 1.2414, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.790308
Epoch 155
Validation binary_cross_entropy = 0.845317
Epoch 156
Validation binary_cross_entropy = 0.915859
Epoch 157
Validation binary_cross_entropy = 0.907971
Epoch 158
Validation binary_cross_entropy = 0.898478
Epoch 159
Loss = 2.3137e-02, PNorm = 96.1553, GNorm = 0.3510, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.872838
Epoch 160
Validation binary_cross_entropy = 0.868915
Epoch 161
Validation binary_cross_entropy = 0.869350
Epoch 162
Validation binary_cross_entropy = 0.924233
Epoch 163
Validation binary_cross_entropy = 0.920203
Epoch 164
Loss = 5.7280e-02, PNorm = 96.3399, GNorm = 1.0009, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.891422
Epoch 165
Validation binary_cross_entropy = 0.824811
Epoch 166
Validation binary_cross_entropy = 0.780703
Epoch 167
Validation binary_cross_entropy = 0.777564
Epoch 168
Validation binary_cross_entropy = 0.806632
Epoch 169
Loss = 2.9773e-02, PNorm = 96.5446, GNorm = 0.4792, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.844092
Epoch 170
Validation binary_cross_entropy = 0.866536
Epoch 171
Validation binary_cross_entropy = 0.889825
Epoch 172
Validation binary_cross_entropy = 0.897344
Epoch 173
Validation binary_cross_entropy = 0.945125
Epoch 174
Loss = 4.7853e-02, PNorm = 96.7330, GNorm = 2.0635, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.022007
Epoch 175
Validation binary_cross_entropy = 1.034281
Epoch 176
Validation binary_cross_entropy = 0.990614
Epoch 177
Validation binary_cross_entropy = 0.958254
Epoch 178
Validation binary_cross_entropy = 0.927534
Epoch 179
Loss = 7.8376e-03, PNorm = 96.9641, GNorm = 0.2216, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.910509
Epoch 180
Validation binary_cross_entropy = 0.908470
Epoch 181
Validation binary_cross_entropy = 0.916866
Epoch 182
Validation binary_cross_entropy = 0.903928
Epoch 183
Validation binary_cross_entropy = 0.885717
Epoch 184
Loss = 2.3623e-02, PNorm = 97.1653, GNorm = 0.4493, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.845217
Epoch 185
Validation binary_cross_entropy = 0.813616
Epoch 186
Validation binary_cross_entropy = 0.798185
Epoch 187
Validation binary_cross_entropy = 0.814206
Epoch 188
Validation binary_cross_entropy = 0.826655
Epoch 189
Loss = 1.6130e-02, PNorm = 97.3724, GNorm = 0.2599, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.824025
Epoch 190
Validation binary_cross_entropy = 0.823330
Epoch 191
Validation binary_cross_entropy = 0.861663
Epoch 192
Validation binary_cross_entropy = 0.878986
Epoch 193
Validation binary_cross_entropy = 0.920926
Epoch 194
Loss = 3.3717e-02, PNorm = 97.5616, GNorm = 0.5975, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.968585
Epoch 195
Validation binary_cross_entropy = 0.904472
Epoch 196
Validation binary_cross_entropy = 0.889484
Epoch 197
Validation binary_cross_entropy = 0.899256
Epoch 198
Validation binary_cross_entropy = 0.952103
Epoch 199
Loss = 3.9661e-02, PNorm = 97.7922, GNorm = 0.7692, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.927057
Epoch 200
Validation binary_cross_entropy = 0.903928
Epoch 201
Validation binary_cross_entropy = 0.869572
Epoch 202
Validation binary_cross_entropy = 0.837924
Epoch 203
Validation binary_cross_entropy = 0.813113
Epoch 204
Loss = 1.3315e-02, PNorm = 98.0375, GNorm = 0.4968, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.794645
Epoch 205
Validation binary_cross_entropy = 0.794279
Epoch 206
Validation binary_cross_entropy = 0.837845
Epoch 207
Validation binary_cross_entropy = 0.919880
Epoch 208
Validation binary_cross_entropy = 1.003221
Epoch 209
Loss = 5.7652e-02, PNorm = 98.2627, GNorm = 0.6221, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.966838
Epoch 210
Validation binary_cross_entropy = 0.951914
Epoch 211
Validation binary_cross_entropy = 0.957064
Epoch 212
Validation binary_cross_entropy = 0.961975
Epoch 213
Validation binary_cross_entropy = 0.976673
Epoch 214
Loss = 5.2270e-02, PNorm = 98.4353, GNorm = 1.1779, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.018026
Epoch 215
Validation binary_cross_entropy = 1.034320
Epoch 216
Validation binary_cross_entropy = 0.961907
Epoch 217
Validation binary_cross_entropy = 0.912633
Epoch 218
Validation binary_cross_entropy = 0.887078
Epoch 219
Loss = 2.3594e-02, PNorm = 98.6056, GNorm = 0.6938, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.873861
Epoch 220
Validation binary_cross_entropy = 0.899234
Epoch 221
Validation binary_cross_entropy = 0.935854
Epoch 222
Validation binary_cross_entropy = 0.927557
Epoch 223
Validation binary_cross_entropy = 0.907311
Epoch 224
Loss = 2.1588e-02, PNorm = 98.7778, GNorm = 0.4855, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.917050
Epoch 225
Validation binary_cross_entropy = 0.932095
Epoch 226
Validation binary_cross_entropy = 0.964059
Epoch 227
Validation binary_cross_entropy = 1.016764
Epoch 228
Validation binary_cross_entropy = 1.082597
Epoch 229
Loss = 3.3726e-02, PNorm = 98.9449, GNorm = 0.7256, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.060782
Epoch 230
Validation binary_cross_entropy = 1.022176
Epoch 231
Validation binary_cross_entropy = 1.013038
Epoch 232
Validation binary_cross_entropy = 1.000611
Epoch 233
Validation binary_cross_entropy = 0.977285
Epoch 234
Loss = 6.0082e-03, PNorm = 99.1083, GNorm = 0.1100, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.959610
Epoch 235
Validation binary_cross_entropy = 0.951520
Epoch 236
Validation binary_cross_entropy = 0.948560
Epoch 237
Validation binary_cross_entropy = 0.975023
Epoch 238
Validation binary_cross_entropy = 1.003326
Epoch 239
Loss = 3.1637e-03, PNorm = 99.2133, GNorm = 0.0891, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.032641
Epoch 240
Validation binary_cross_entropy = 1.070192
Epoch 241
Validation binary_cross_entropy = 1.104726
Epoch 242
Validation binary_cross_entropy = 1.110985
Epoch 243
Validation binary_cross_entropy = 1.107358
Epoch 244
Loss = 1.5087e-02, PNorm = 99.3201, GNorm = 0.5945, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.088846
Epoch 245
Validation binary_cross_entropy = 1.090456
Epoch 246
Validation binary_cross_entropy = 1.101933
Epoch 247
Validation binary_cross_entropy = 1.111756
Epoch 248
Validation binary_cross_entropy = 1.124328
Epoch 249
Loss = 6.4558e-03, PNorm = 99.4250, GNorm = 0.1428, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.143260
Epoch 250
Validation binary_cross_entropy = 1.143088
Epoch 251
Validation binary_cross_entropy = 1.152035
Epoch 252
Validation binary_cross_entropy = 1.143633
Epoch 253
Validation binary_cross_entropy = 1.105185
Epoch 254
Loss = 6.3783e-03, PNorm = 99.5247, GNorm = 0.3969, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.085207
Epoch 255
Validation binary_cross_entropy = 1.074061
Epoch 256
Validation binary_cross_entropy = 1.063283
Epoch 257
Validation binary_cross_entropy = 1.092559
Epoch 258
Validation binary_cross_entropy = 1.090871
Epoch 259
Loss = 2.6331e-02, PNorm = 99.6339, GNorm = 0.8629, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.090493
Epoch 260
Validation binary_cross_entropy = 1.127526
Epoch 261
Validation binary_cross_entropy = 1.148704
Epoch 262
Validation binary_cross_entropy = 1.189291
Epoch 263
Validation binary_cross_entropy = 1.201985
Epoch 264
Loss = 2.2129e-02, PNorm = 99.7606, GNorm = 0.1712, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.228437
Epoch 265
Validation binary_cross_entropy = 1.228556
Epoch 266
Validation binary_cross_entropy = 1.162404
Epoch 267
Validation binary_cross_entropy = 1.133275
Epoch 268
Validation binary_cross_entropy = 1.113217
Epoch 269
Loss = 1.2338e-02, PNorm = 99.8602, GNorm = 0.1155, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.077979
Epoch 270
Validation binary_cross_entropy = 1.045475
Epoch 271
Validation binary_cross_entropy = 1.039039
Epoch 272
Validation binary_cross_entropy = 1.018019
Epoch 273
Validation binary_cross_entropy = 1.019365
Epoch 274
Loss = 2.0876e-03, PNorm = 99.9385, GNorm = 0.0517, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.022811
Epoch 275
Validation binary_cross_entropy = 1.040565
Epoch 276
Validation binary_cross_entropy = 1.060937
Epoch 277
Validation binary_cross_entropy = 1.077052
Epoch 278
Validation binary_cross_entropy = 1.116714
Epoch 279
Loss = 1.5768e-02, PNorm = 100.0237, GNorm = 0.1294, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.143617
Epoch 280
Validation binary_cross_entropy = 1.147569
Epoch 281
Validation binary_cross_entropy = 1.152321
Epoch 282
Validation binary_cross_entropy = 1.165738
Epoch 283
Validation binary_cross_entropy = 1.173608
Epoch 284
Loss = 1.4029e-02, PNorm = 100.1233, GNorm = 0.5369, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.185670
Epoch 285
Validation binary_cross_entropy = 1.191649
Epoch 286
Validation binary_cross_entropy = 1.220978
Epoch 287
Validation binary_cross_entropy = 1.226177
Epoch 288
Validation binary_cross_entropy = 1.244802
Epoch 289
Loss = 6.9702e-03, PNorm = 100.2042, GNorm = 0.3304, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.241845
Epoch 290
Validation binary_cross_entropy = 1.219592
Epoch 291
Validation binary_cross_entropy = 1.197394
Epoch 292
Validation binary_cross_entropy = 1.184429
Epoch 293
Validation binary_cross_entropy = 1.186021
Epoch 294
Loss = 1.3980e-03, PNorm = 100.2812, GNorm = 0.0232, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.190269
Epoch 295
Validation binary_cross_entropy = 1.201336
Epoch 296
Validation binary_cross_entropy = 1.215091
Epoch 297
Validation binary_cross_entropy = 1.217068
Epoch 298
Validation binary_cross_entropy = 1.232680
Epoch 299
Loss = 9.9891e-03, PNorm = 100.3381, GNorm = 0.4065, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.222027
Model 0 best validation binary_cross_entropy = 0.288000 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.245994
Ensemble test binary_cross_entropy = 0.245994
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.441464
Epoch 1
Validation binary_cross_entropy = 0.470540
Epoch 2
Validation binary_cross_entropy = 0.533845
Epoch 3
Validation binary_cross_entropy = 0.319231
Epoch 4
Loss = 5.3128e-01, PNorm = 87.7844, GNorm = 3.1196, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.432426
Epoch 5
Validation binary_cross_entropy = 0.402943
Epoch 6
Validation binary_cross_entropy = 0.341180
Epoch 7
Validation binary_cross_entropy = 0.550505
Epoch 8
Validation binary_cross_entropy = 0.401078
Epoch 9
Loss = 3.3894e-01, PNorm = 88.1807, GNorm = 1.9355, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.484490
Epoch 10
Validation binary_cross_entropy = 0.377124
Epoch 11
Validation binary_cross_entropy = 0.409850
Epoch 12
Validation binary_cross_entropy = 0.321846
Epoch 13
Validation binary_cross_entropy = 0.412538
Epoch 14
Loss = 3.8596e-01, PNorm = 88.5808, GNorm = 2.3853, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.402301
Epoch 15
Validation binary_cross_entropy = 0.320119
Epoch 16
Validation binary_cross_entropy = 0.365360
Epoch 17
Validation binary_cross_entropy = 0.371487
Epoch 18
Validation binary_cross_entropy = 0.340209
Epoch 19
Loss = 2.9777e-01, PNorm = 88.8652, GNorm = 2.2210, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.312963
Epoch 20
Validation binary_cross_entropy = 0.376925
Epoch 21
Validation binary_cross_entropy = 0.388792
Epoch 22
Validation binary_cross_entropy = 0.333820
Epoch 23
Validation binary_cross_entropy = 0.334788
Epoch 24
Loss = 2.5069e-01, PNorm = 89.0914, GNorm = 2.5617, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.361003
Epoch 25
Validation binary_cross_entropy = 0.496963
Epoch 26
Validation binary_cross_entropy = 0.355493
Epoch 27
Validation binary_cross_entropy = 0.342426
Epoch 28
Validation binary_cross_entropy = 0.366089
Epoch 29
Loss = 2.5917e-01, PNorm = 89.2976, GNorm = 1.5496, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.434963
Epoch 30
Validation binary_cross_entropy = 0.382107
Epoch 31
Validation binary_cross_entropy = 0.361328
Epoch 32
Validation binary_cross_entropy = 0.348572
Epoch 33
Validation binary_cross_entropy = 0.348695
Epoch 34
Loss = 2.8618e-01, PNorm = 89.5287, GNorm = 2.0118, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.355723
Epoch 35
Validation binary_cross_entropy = 0.365858
Epoch 36
Validation binary_cross_entropy = 0.421995
Epoch 37
Validation binary_cross_entropy = 0.375054
Epoch 38
Validation binary_cross_entropy = 0.369874
Epoch 39
Loss = 2.8088e-01, PNorm = 89.7448, GNorm = 2.0041, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.370580
Epoch 40
Validation binary_cross_entropy = 0.372420
Epoch 41
Validation binary_cross_entropy = 0.357049
Epoch 42
Validation binary_cross_entropy = 0.382567
Epoch 43
Validation binary_cross_entropy = 0.357848
Epoch 44
Loss = 1.3402e-01, PNorm = 89.9943, GNorm = 1.9261, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.365648
Epoch 45
Validation binary_cross_entropy = 0.378948
Epoch 46
Validation binary_cross_entropy = 0.369645
Epoch 47
Validation binary_cross_entropy = 0.370062
Epoch 48
Validation binary_cross_entropy = 0.361115
Epoch 49
Loss = 1.7852e-01, PNorm = 90.2165, GNorm = 1.2824, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.374508
Epoch 50
Validation binary_cross_entropy = 0.418846
Epoch 51
Validation binary_cross_entropy = 0.358722
Epoch 52
Validation binary_cross_entropy = 0.380725
Epoch 53
Validation binary_cross_entropy = 0.390449
Epoch 54
Loss = 1.3737e-01, PNorm = 90.4566, GNorm = 1.4033, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.413260
Epoch 55
Validation binary_cross_entropy = 0.405201
Epoch 56
Validation binary_cross_entropy = 0.381606
Epoch 57
Validation binary_cross_entropy = 0.375860
Epoch 58
Validation binary_cross_entropy = 0.426657
Epoch 59
Loss = 2.2800e-01, PNorm = 90.7035, GNorm = 2.2631, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.393584
Epoch 60
Validation binary_cross_entropy = 0.377175
Epoch 61
Validation binary_cross_entropy = 0.391431
Epoch 62
Validation binary_cross_entropy = 0.398182
Epoch 63
Validation binary_cross_entropy = 0.406499
Epoch 64
Loss = 1.7914e-01, PNorm = 90.9753, GNorm = 1.3233, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.410279
Epoch 65
Validation binary_cross_entropy = 0.410237
Epoch 66
Validation binary_cross_entropy = 0.420305
Epoch 67
Validation binary_cross_entropy = 0.442516
Epoch 68
Validation binary_cross_entropy = 0.427387
Epoch 69
Loss = 1.4325e-01, PNorm = 91.2328, GNorm = 0.8490, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.425142
Epoch 70
Validation binary_cross_entropy = 0.419524
Epoch 71
Validation binary_cross_entropy = 0.414613
Epoch 72
Validation binary_cross_entropy = 0.418396
Epoch 73
Validation binary_cross_entropy = 0.430468
Epoch 74
Loss = 9.9239e-02, PNorm = 91.5550, GNorm = 1.9324, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.474157
Epoch 75
Validation binary_cross_entropy = 0.498777
Epoch 76
Validation binary_cross_entropy = 0.479115
Epoch 77
Validation binary_cross_entropy = 0.458548
Epoch 78
Validation binary_cross_entropy = 0.442062
Epoch 79
Loss = 9.7864e-02, PNorm = 91.9061, GNorm = 1.2614, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.460811
Epoch 80
Validation binary_cross_entropy = 0.449343
Epoch 81
Validation binary_cross_entropy = 0.438620
Epoch 82
Validation binary_cross_entropy = 0.435636
Epoch 83
Validation binary_cross_entropy = 0.427381
Epoch 84
Loss = 8.0763e-02, PNorm = 92.2093, GNorm = 1.0861, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.431939
Epoch 85
Validation binary_cross_entropy = 0.454094
Epoch 86
Validation binary_cross_entropy = 0.473584
Epoch 87
Validation binary_cross_entropy = 0.501438
Epoch 88
Validation binary_cross_entropy = 0.503096
Epoch 89
Loss = 6.5803e-02, PNorm = 92.5021, GNorm = 1.0315, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.525353
Epoch 90
Validation binary_cross_entropy = 0.501428
Epoch 91
Validation binary_cross_entropy = 0.495053
Epoch 92
Validation binary_cross_entropy = 0.500912
Epoch 93
Validation binary_cross_entropy = 0.511432
Epoch 94
Loss = 9.4865e-02, PNorm = 92.7942, GNorm = 1.5111, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.536454
Epoch 95
Validation binary_cross_entropy = 0.531254
Epoch 96
Validation binary_cross_entropy = 0.523977
Epoch 97
Validation binary_cross_entropy = 0.534463
Epoch 98
Validation binary_cross_entropy = 0.532222
Epoch 99
Loss = 8.2687e-02, PNorm = 93.0471, GNorm = 0.6471, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.538311
Epoch 100
Validation binary_cross_entropy = 0.552825
Epoch 101
Validation binary_cross_entropy = 0.555082
Epoch 102
Validation binary_cross_entropy = 0.551946
Epoch 103
Validation binary_cross_entropy = 0.557529
Epoch 104
Loss = 4.9016e-02, PNorm = 93.3096, GNorm = 0.5075, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.574491
Epoch 105
Validation binary_cross_entropy = 0.553800
Epoch 106
Validation binary_cross_entropy = 0.572146
Epoch 107
Validation binary_cross_entropy = 0.551513
Epoch 108
Validation binary_cross_entropy = 0.596451
Epoch 109
Loss = 1.4335e-01, PNorm = 93.5338, GNorm = 1.7795, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.589299
Epoch 110
Validation binary_cross_entropy = 0.539831
Epoch 111
Validation binary_cross_entropy = 0.534098
Epoch 112
Validation binary_cross_entropy = 0.549558
Epoch 113
Validation binary_cross_entropy = 0.597824
Epoch 114
Loss = 1.2243e-01, PNorm = 93.8117, GNorm = 1.7130, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.630256
Epoch 115
Validation binary_cross_entropy = 0.630032
Epoch 116
Validation binary_cross_entropy = 0.643625
Epoch 117
Validation binary_cross_entropy = 0.651238
Epoch 118
Validation binary_cross_entropy = 0.660747
Epoch 119
Loss = 9.1652e-02, PNorm = 94.0547, GNorm = 2.5033, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.650488
Epoch 120
Validation binary_cross_entropy = 0.579323
Epoch 121
Validation binary_cross_entropy = 0.582653
Epoch 122
Validation binary_cross_entropy = 0.604022
Epoch 123
Validation binary_cross_entropy = 0.631872
Epoch 124
Loss = 5.1183e-02, PNorm = 94.3360, GNorm = 0.6474, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.646980
Epoch 125
Validation binary_cross_entropy = 0.624475
Epoch 126
Validation binary_cross_entropy = 0.621042
Epoch 127
Validation binary_cross_entropy = 0.622931
Epoch 128
Validation binary_cross_entropy = 0.668530
Epoch 129
Loss = 9.4151e-02, PNorm = 94.5776, GNorm = 0.7505, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.611750
Epoch 130
Validation binary_cross_entropy = 0.590689
Epoch 131
Validation binary_cross_entropy = 0.602663
Epoch 132
Validation binary_cross_entropy = 0.643718
Epoch 133
Validation binary_cross_entropy = 0.667139
Epoch 134
Loss = 6.1495e-02, PNorm = 94.8076, GNorm = 0.8898, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.653525
Epoch 135
Validation binary_cross_entropy = 0.614459
Epoch 136
Validation binary_cross_entropy = 0.604249
Epoch 137
Validation binary_cross_entropy = 0.649414
Epoch 138
Validation binary_cross_entropy = 0.686191
Epoch 139
Loss = 5.4995e-02, PNorm = 95.0445, GNorm = 0.2233, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.682302
Epoch 140
Validation binary_cross_entropy = 0.719788
Epoch 141
Validation binary_cross_entropy = 0.673096
Epoch 142
Validation binary_cross_entropy = 0.686063
Epoch 143
Validation binary_cross_entropy = 0.719897
Epoch 144
Loss = 6.2558e-02, PNorm = 95.2738, GNorm = 1.4431, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.712080
Epoch 145
Validation binary_cross_entropy = 0.684589
Epoch 146
Validation binary_cross_entropy = 0.697564
Epoch 147
Validation binary_cross_entropy = 0.730645
Epoch 148
Validation binary_cross_entropy = 0.779244
Epoch 149
Loss = 1.2473e-01, PNorm = 95.5117, GNorm = 2.0258, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.775975
Epoch 150
Validation binary_cross_entropy = 0.736656
Epoch 151
Validation binary_cross_entropy = 0.729368
Epoch 152
Validation binary_cross_entropy = 0.733492
Epoch 153
Validation binary_cross_entropy = 0.734023
Epoch 154
Loss = 7.6812e-02, PNorm = 95.7106, GNorm = 1.3263, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.705966
Epoch 155
Validation binary_cross_entropy = 0.687062
Epoch 156
Validation binary_cross_entropy = 0.689302
Epoch 157
Validation binary_cross_entropy = 0.707840
Epoch 158
Validation binary_cross_entropy = 0.737882
Epoch 159
Loss = 1.5654e-01, PNorm = 95.8894, GNorm = 1.7948, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.722106
Epoch 160
Validation binary_cross_entropy = 0.673653
Epoch 161
Validation binary_cross_entropy = 0.655551
Epoch 162
Validation binary_cross_entropy = 0.664061
Epoch 163
Validation binary_cross_entropy = 0.694010
Epoch 164
Loss = 5.2356e-02, PNorm = 96.1178, GNorm = 1.3747, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.697513
Epoch 165
Validation binary_cross_entropy = 0.683101
Epoch 166
Validation binary_cross_entropy = 0.690432
Epoch 167
Validation binary_cross_entropy = 0.683865
Epoch 168
Validation binary_cross_entropy = 0.749660
Epoch 169
Loss = 8.9486e-02, PNorm = 96.3524, GNorm = 2.6846, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.765042
Epoch 170
Validation binary_cross_entropy = 0.712126
Epoch 171
Validation binary_cross_entropy = 0.716129
Epoch 172
Validation binary_cross_entropy = 0.730164
Epoch 173
Validation binary_cross_entropy = 0.770705
Epoch 174
Loss = 6.6396e-02, PNorm = 96.5658, GNorm = 1.0265, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.823232
Epoch 175
Validation binary_cross_entropy = 0.774723
Epoch 176
Validation binary_cross_entropy = 0.727251
Epoch 177
Validation binary_cross_entropy = 0.707263
Epoch 178
Validation binary_cross_entropy = 0.696140
Epoch 179
Loss = 3.1400e-02, PNorm = 96.7664, GNorm = 0.1263, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.704089
Epoch 180
Validation binary_cross_entropy = 0.718171
Epoch 181
Validation binary_cross_entropy = 0.734800
Epoch 182
Validation binary_cross_entropy = 0.778756
Epoch 183
Validation binary_cross_entropy = 0.778640
Epoch 184
Loss = 4.8804e-02, PNorm = 96.9244, GNorm = 0.7972, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.776439
Epoch 185
Validation binary_cross_entropy = 0.783411
Epoch 186
Validation binary_cross_entropy = 0.798060
Epoch 187
Validation binary_cross_entropy = 0.833941
Epoch 188
Validation binary_cross_entropy = 0.882076
Epoch 189
Loss = 4.5071e-02, PNorm = 97.0812, GNorm = 0.5933, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.955787
Epoch 190
Validation binary_cross_entropy = 0.912837
Epoch 191
Validation binary_cross_entropy = 0.865616
Epoch 192
Validation binary_cross_entropy = 0.841644
Epoch 193
Validation binary_cross_entropy = 0.832261
Epoch 194
Loss = 4.2149e-02, PNorm = 97.2141, GNorm = 0.7864, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.845183
Epoch 195
Validation binary_cross_entropy = 0.862032
Epoch 196
Validation binary_cross_entropy = 0.858696
Epoch 197
Validation binary_cross_entropy = 0.834772
Epoch 198
Validation binary_cross_entropy = 0.816295
Epoch 199
Loss = 1.4233e-02, PNorm = 97.3457, GNorm = 0.1340, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.806410
Epoch 200
Validation binary_cross_entropy = 0.814904
Epoch 201
Validation binary_cross_entropy = 0.837226
Epoch 202
Validation binary_cross_entropy = 0.886384
Epoch 203
Validation binary_cross_entropy = 0.974517
Epoch 204
Loss = 5.8859e-02, PNorm = 97.4981, GNorm = 1.4539, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.991070
Epoch 205
Validation binary_cross_entropy = 0.951617
Epoch 206
Validation binary_cross_entropy = 0.937850
Epoch 207
Validation binary_cross_entropy = 0.965098
Epoch 208
Validation binary_cross_entropy = 1.038897
Epoch 209
Loss = 4.7925e-02, PNorm = 97.6723, GNorm = 0.8330, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.042716
Epoch 210
Validation binary_cross_entropy = 0.971141
Epoch 211
Validation binary_cross_entropy = 0.861791
Epoch 212
Validation binary_cross_entropy = 0.800363
Epoch 213
Validation binary_cross_entropy = 0.757316
Epoch 214
Loss = 3.2910e-02, PNorm = 97.8954, GNorm = 0.1701, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.761606
Epoch 215
Validation binary_cross_entropy = 0.797992
Epoch 216
Validation binary_cross_entropy = 0.776462
Epoch 217
Validation binary_cross_entropy = 0.756555
Epoch 218
Validation binary_cross_entropy = 0.752078
Epoch 219
Loss = 1.5353e-02, PNorm = 98.0825, GNorm = 0.4147, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.765301
Epoch 220
Validation binary_cross_entropy = 0.792900
Epoch 221
Validation binary_cross_entropy = 0.811626
Epoch 222
Validation binary_cross_entropy = 0.866347
Epoch 223
Validation binary_cross_entropy = 0.914571
Epoch 224
Loss = 1.0032e-02, PNorm = 98.2288, GNorm = 0.3491, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.935519
Epoch 225
Validation binary_cross_entropy = 0.940335
Epoch 226
Validation binary_cross_entropy = 0.917003
Epoch 227
Validation binary_cross_entropy = 0.889238
Epoch 228
Validation binary_cross_entropy = 0.879715
Epoch 229
Loss = 2.7424e-02, PNorm = 98.3419, GNorm = 0.2373, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.898889
Epoch 230
Validation binary_cross_entropy = 0.901494
Epoch 231
Validation binary_cross_entropy = 0.883279
Epoch 232
Validation binary_cross_entropy = 0.879532
Epoch 233
Validation binary_cross_entropy = 0.863420
Epoch 234
Loss = 1.5268e-02, PNorm = 98.4478, GNorm = 0.5595, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.866336
Epoch 235
Validation binary_cross_entropy = 0.894931
Epoch 236
Validation binary_cross_entropy = 0.905704
Epoch 237
Validation binary_cross_entropy = 0.915478
Epoch 238
Validation binary_cross_entropy = 0.924334
Epoch 239
Loss = 7.7516e-03, PNorm = 98.5597, GNorm = 0.3153, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.922737
Epoch 240
Validation binary_cross_entropy = 0.933637
Epoch 241
Validation binary_cross_entropy = 0.954339
Epoch 242
Validation binary_cross_entropy = 0.972112
Epoch 243
Validation binary_cross_entropy = 1.016789
Epoch 244
Loss = 1.4616e-02, PNorm = 98.6651, GNorm = 0.6935, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.038607
Epoch 245
Validation binary_cross_entropy = 1.038546
Epoch 246
Validation binary_cross_entropy = 1.044228
Epoch 247
Validation binary_cross_entropy = 1.048619
Epoch 248
Validation binary_cross_entropy = 1.044031
Epoch 249
Loss = 2.4881e-03, PNorm = 98.7341, GNorm = 0.0205, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.037483
Epoch 250
Validation binary_cross_entropy = 1.061836
Epoch 251
Validation binary_cross_entropy = 1.097019
Epoch 252
Validation binary_cross_entropy = 1.112006
Epoch 253
Validation binary_cross_entropy = 1.106692
Epoch 254
Loss = 7.2722e-03, PNorm = 98.7995, GNorm = 0.1051, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.087842
Epoch 255
Validation binary_cross_entropy = 1.091416
Epoch 256
Validation binary_cross_entropy = 1.111379
Epoch 257
Validation binary_cross_entropy = 1.132244
Epoch 258
Validation binary_cross_entropy = 1.146479
Epoch 259
Loss = 7.2586e-03, PNorm = 98.8705, GNorm = 0.2310, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.160398
Epoch 260
Validation binary_cross_entropy = 1.159596
Epoch 261
Validation binary_cross_entropy = 1.158721
Epoch 262
Validation binary_cross_entropy = 1.154293
Epoch 263
Validation binary_cross_entropy = 1.148015
Epoch 264
Loss = 6.8084e-03, PNorm = 98.9129, GNorm = 0.2476, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.131335
Epoch 265
Validation binary_cross_entropy = 1.124325
Epoch 266
Validation binary_cross_entropy = 1.135808
Epoch 267
Validation binary_cross_entropy = 1.159282
Epoch 268
Validation binary_cross_entropy = 1.141852
Epoch 269
Loss = 4.8924e-03, PNorm = 98.9671, GNorm = 0.1396, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.115772
Epoch 270
Validation binary_cross_entropy = 1.093779
Epoch 271
Validation binary_cross_entropy = 1.085166
Epoch 272
Validation binary_cross_entropy = 1.076402
Epoch 273
Validation binary_cross_entropy = 1.068074
Epoch 274
Loss = 3.8191e-03, PNorm = 99.0338, GNorm = 0.2387, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.068026
Epoch 275
Validation binary_cross_entropy = 1.080755
Epoch 276
Validation binary_cross_entropy = 1.094850
Epoch 277
Validation binary_cross_entropy = 1.094561
Epoch 278
Validation binary_cross_entropy = 1.088389
Epoch 279
Loss = 2.5222e-03, PNorm = 99.0970, GNorm = 0.1063, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.082187
Epoch 280
Validation binary_cross_entropy = 1.075833
Epoch 281
Validation binary_cross_entropy = 1.077397
Epoch 282
Validation binary_cross_entropy = 1.088146
Epoch 283
Validation binary_cross_entropy = 1.099463
Epoch 284
Loss = 1.5127e-02, PNorm = 99.1553, GNorm = 0.7860, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.111730
Epoch 285
Validation binary_cross_entropy = 1.124416
Epoch 286
Validation binary_cross_entropy = 1.133787
Epoch 287
Validation binary_cross_entropy = 1.140167
Epoch 288
Validation binary_cross_entropy = 1.144799
Epoch 289
Loss = 8.0309e-03, PNorm = 99.2300, GNorm = 0.2245, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.157035
Epoch 290
Validation binary_cross_entropy = 1.177589
Epoch 291
Validation binary_cross_entropy = 1.205809
Epoch 292
Validation binary_cross_entropy = 1.223191
Epoch 293
Validation binary_cross_entropy = 1.225157
Epoch 294
Loss = 2.5764e-03, PNorm = 99.2937, GNorm = 0.0594, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.232026
Epoch 295
Validation binary_cross_entropy = 1.228570
Epoch 296
Validation binary_cross_entropy = 1.245261
Epoch 297
Validation binary_cross_entropy = 1.274658
Epoch 298
Validation binary_cross_entropy = 1.274432
Epoch 299
Loss = 1.1928e-03, PNorm = 99.3521, GNorm = 0.0199, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.259319
Model 0 best validation binary_cross_entropy = 0.312963 on epoch 19
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.161298
Ensemble test binary_cross_entropy = 0.161298
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.636528
Epoch 1
Validation binary_cross_entropy = 0.577725
Epoch 2
Validation binary_cross_entropy = 0.331408
Epoch 3
Validation binary_cross_entropy = 0.302955
Epoch 4
Loss = 5.6908e-01, PNorm = 87.7979, GNorm = 3.1838, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.447177
Epoch 5
Validation binary_cross_entropy = 0.367537
Epoch 6
Validation binary_cross_entropy = 0.380994
Epoch 7
Validation binary_cross_entropy = 0.577485
Epoch 8
Validation binary_cross_entropy = 0.307268
Epoch 9
Loss = 4.3571e-01, PNorm = 88.2074, GNorm = 1.5596, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.397333
Epoch 10
Validation binary_cross_entropy = 0.439937
Epoch 11
Validation binary_cross_entropy = 0.390716
Epoch 12
Validation binary_cross_entropy = 0.320493
Epoch 13
Validation binary_cross_entropy = 0.379088
Epoch 14
Loss = 4.3377e-01, PNorm = 88.6066, GNorm = 2.2275, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.417966
Epoch 15
Validation binary_cross_entropy = 0.366165
Epoch 16
Validation binary_cross_entropy = 0.346919
Epoch 17
Validation binary_cross_entropy = 0.348010
Epoch 18
Validation binary_cross_entropy = 0.333565
Epoch 19
Loss = 3.2341e-01, PNorm = 88.8920, GNorm = 1.6929, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.356860
Epoch 20
Validation binary_cross_entropy = 0.472365
Epoch 21
Validation binary_cross_entropy = 0.341573
Epoch 22
Validation binary_cross_entropy = 0.316623
Epoch 23
Validation binary_cross_entropy = 0.484823
Epoch 24
Loss = 3.8390e-01, PNorm = 89.1105, GNorm = 3.4665, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.437040
Epoch 25
Validation binary_cross_entropy = 0.323281
Epoch 26
Validation binary_cross_entropy = 0.339976
Epoch 27
Validation binary_cross_entropy = 0.503481
Epoch 28
Validation binary_cross_entropy = 0.445826
Epoch 29
Loss = 2.3919e-01, PNorm = 89.3157, GNorm = 1.7059, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.340748
Epoch 30
Validation binary_cross_entropy = 0.333295
Epoch 31
Validation binary_cross_entropy = 0.353804
Epoch 32
Validation binary_cross_entropy = 0.457945
Epoch 33
Validation binary_cross_entropy = 0.459504
Epoch 34
Loss = 1.9912e-01, PNorm = 89.5192, GNorm = 1.6249, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.383948
Epoch 35
Validation binary_cross_entropy = 0.354491
Epoch 36
Validation binary_cross_entropy = 0.365653
Epoch 37
Validation binary_cross_entropy = 0.415816
Epoch 38
Validation binary_cross_entropy = 0.427536
Epoch 39
Loss = 3.0040e-01, PNorm = 89.7140, GNorm = 1.8736, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.389332
Epoch 40
Validation binary_cross_entropy = 0.406352
Epoch 41
Validation binary_cross_entropy = 0.403509
Epoch 42
Validation binary_cross_entropy = 0.395534
Epoch 43
Validation binary_cross_entropy = 0.423229
Epoch 44
Loss = 2.9296e-01, PNorm = 89.9214, GNorm = 2.3658, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.387440
Epoch 45
Validation binary_cross_entropy = 0.384162
Epoch 46
Validation binary_cross_entropy = 0.385001
Epoch 47
Validation binary_cross_entropy = 0.377907
Epoch 48
Validation binary_cross_entropy = 0.384637
Epoch 49
Loss = 1.1071e-01, PNorm = 90.1951, GNorm = 0.7235, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.427828
Epoch 50
Validation binary_cross_entropy = 0.452396
Epoch 51
Validation binary_cross_entropy = 0.415761
Epoch 52
Validation binary_cross_entropy = 0.425471
Epoch 53
Validation binary_cross_entropy = 0.433222
Epoch 54
Loss = 1.6182e-01, PNorm = 90.4923, GNorm = 1.8716, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.439059
Epoch 55
Validation binary_cross_entropy = 0.431534
Epoch 56
Validation binary_cross_entropy = 0.427305
Epoch 57
Validation binary_cross_entropy = 0.427328
Epoch 58
Validation binary_cross_entropy = 0.433911
Epoch 59
Loss = 7.4242e-02, PNorm = 90.7828, GNorm = 0.6091, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.483407
Epoch 60
Validation binary_cross_entropy = 0.441586
Epoch 61
Validation binary_cross_entropy = 0.429380
Epoch 62
Validation binary_cross_entropy = 0.427431
Epoch 63
Validation binary_cross_entropy = 0.422403
Epoch 64
Loss = 9.8158e-02, PNorm = 91.0506, GNorm = 0.7999, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.427978
Epoch 65
Validation binary_cross_entropy = 0.426872
Epoch 66
Validation binary_cross_entropy = 0.440801
Epoch 67
Validation binary_cross_entropy = 0.450794
Epoch 68
Validation binary_cross_entropy = 0.456991
Epoch 69
Loss = 1.2147e-01, PNorm = 91.3259, GNorm = 1.6305, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.458947
Epoch 70
Validation binary_cross_entropy = 0.462703
Epoch 71
Validation binary_cross_entropy = 0.465922
Epoch 72
Validation binary_cross_entropy = 0.469379
Epoch 73
Validation binary_cross_entropy = 0.490810
Epoch 74
Loss = 1.0451e-01, PNorm = 91.5841, GNorm = 1.1826, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.497467
Epoch 75
Validation binary_cross_entropy = 0.524842
Epoch 76
Validation binary_cross_entropy = 0.524109
Epoch 77
Validation binary_cross_entropy = 0.522756
Epoch 78
Validation binary_cross_entropy = 0.502087
Epoch 79
Loss = 5.5554e-02, PNorm = 91.8694, GNorm = 0.6885, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.501166
Epoch 80
Validation binary_cross_entropy = 0.515762
Epoch 81
Validation binary_cross_entropy = 0.545672
Epoch 82
Validation binary_cross_entropy = 0.568073
Epoch 83
Validation binary_cross_entropy = 0.607683
Epoch 84
Loss = 1.2448e-01, PNorm = 92.1520, GNorm = 2.8279, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.597900
Epoch 85
Validation binary_cross_entropy = 0.641873
Epoch 86
Validation binary_cross_entropy = 0.598091
Epoch 87
Validation binary_cross_entropy = 0.586392
Epoch 88
Validation binary_cross_entropy = 0.602263
Epoch 89
Loss = 1.4394e-01, PNorm = 92.4599, GNorm = 1.3180, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.592846
Epoch 90
Validation binary_cross_entropy = 0.556627
Epoch 91
Validation binary_cross_entropy = 0.532429
Epoch 92
Validation binary_cross_entropy = 0.517505
Epoch 93
Validation binary_cross_entropy = 0.510877
Epoch 94
Loss = 1.2936e-01, PNorm = 92.7492, GNorm = 1.9172, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.497832
Epoch 95
Validation binary_cross_entropy = 0.496436
Epoch 96
Validation binary_cross_entropy = 0.524273
Epoch 97
Validation binary_cross_entropy = 0.562704
Epoch 98
Validation binary_cross_entropy = 0.607393
Epoch 99
Loss = 1.1242e-01, PNorm = 93.0612, GNorm = 0.5920, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.605331
Epoch 100
Validation binary_cross_entropy = 0.611483
Epoch 101
Validation binary_cross_entropy = 0.598521
Epoch 102
Validation binary_cross_entropy = 0.557587
Epoch 103
Validation binary_cross_entropy = 0.538863
Epoch 104
Loss = 9.2182e-02, PNorm = 93.3351, GNorm = 2.0042, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.527858
Epoch 105
Validation binary_cross_entropy = 0.535562
Epoch 106
Validation binary_cross_entropy = 0.559235
Epoch 107
Validation binary_cross_entropy = 0.567228
Epoch 108
Validation binary_cross_entropy = 0.605922
Epoch 109
Loss = 7.0870e-02, PNorm = 93.6179, GNorm = 1.2601, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.623300
Epoch 110
Validation binary_cross_entropy = 0.625416
Epoch 111
Validation binary_cross_entropy = 0.626936
Epoch 112
Validation binary_cross_entropy = 0.610467
Epoch 113
Validation binary_cross_entropy = 0.580796
Epoch 114
Loss = 6.7859e-02, PNorm = 93.8709, GNorm = 1.3407, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.572137
Epoch 115
Validation binary_cross_entropy = 0.578553
Epoch 116
Validation binary_cross_entropy = 0.590809
Epoch 117
Validation binary_cross_entropy = 0.629699
Epoch 118
Validation binary_cross_entropy = 0.655644
Epoch 119
Loss = 3.7812e-02, PNorm = 94.1273, GNorm = 0.5408, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.635807
Epoch 120
Validation binary_cross_entropy = 0.639088
Epoch 121
Validation binary_cross_entropy = 0.638175
Epoch 122
Validation binary_cross_entropy = 0.597402
Epoch 123
Validation binary_cross_entropy = 0.583092
Epoch 124
Loss = 6.1311e-02, PNorm = 94.3386, GNorm = 0.5227, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.596873
Epoch 125
Validation binary_cross_entropy = 0.620182
Epoch 126
Validation binary_cross_entropy = 0.670527
Epoch 127
Validation binary_cross_entropy = 0.710927
Epoch 128
Validation binary_cross_entropy = 0.766254
Epoch 129
Loss = 9.6836e-02, PNorm = 94.5980, GNorm = 1.3597, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.794879
Epoch 130
Validation binary_cross_entropy = 0.818342
Epoch 131
Validation binary_cross_entropy = 0.720211
Epoch 132
Validation binary_cross_entropy = 0.632186
Epoch 133
Validation binary_cross_entropy = 0.608175
Epoch 134
Loss = 5.1366e-02, PNorm = 94.8785, GNorm = 0.5095, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.563737
Epoch 135
Validation binary_cross_entropy = 0.573023
Epoch 136
Validation binary_cross_entropy = 0.593230
Epoch 137
Validation binary_cross_entropy = 0.600481
Epoch 138
Validation binary_cross_entropy = 0.611359
Epoch 139
Loss = 5.7677e-02, PNorm = 95.1833, GNorm = 0.5915, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.595847
Epoch 140
Validation binary_cross_entropy = 0.592196
Epoch 141
Validation binary_cross_entropy = 0.594123
Epoch 142
Validation binary_cross_entropy = 0.603148
Epoch 143
Validation binary_cross_entropy = 0.625121
Epoch 144
Loss = 2.6754e-02, PNorm = 95.4437, GNorm = 1.0038, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.653826
Epoch 145
Validation binary_cross_entropy = 0.659696
Epoch 146
Validation binary_cross_entropy = 0.675984
Epoch 147
Validation binary_cross_entropy = 0.698727
Epoch 148
Validation binary_cross_entropy = 0.728645
Epoch 149
Loss = 9.6672e-02, PNorm = 95.6551, GNorm = 2.0939, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.746432
Epoch 150
Validation binary_cross_entropy = 0.816481
Epoch 151
Validation binary_cross_entropy = 0.760100
Epoch 152
Validation binary_cross_entropy = 0.764527
Epoch 153
Validation binary_cross_entropy = 0.743704
Epoch 154
Loss = 1.0936e-02, PNorm = 95.8601, GNorm = 0.1790, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.752683
Epoch 155
Validation binary_cross_entropy = 0.775905
Epoch 156
Validation binary_cross_entropy = 0.809718
Epoch 157
Validation binary_cross_entropy = 0.739356
Epoch 158
Validation binary_cross_entropy = 0.709625
Epoch 159
Loss = 5.6898e-02, PNorm = 96.0691, GNorm = 1.2124, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.678850
Epoch 160
Validation binary_cross_entropy = 0.665543
Epoch 161
Validation binary_cross_entropy = 0.662517
Epoch 162
Validation binary_cross_entropy = 0.651521
Epoch 163
Validation binary_cross_entropy = 0.645878
Epoch 164
Loss = 4.7272e-02, PNorm = 96.3136, GNorm = 0.2031, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.649910
Epoch 165
Validation binary_cross_entropy = 0.659248
Epoch 166
Validation binary_cross_entropy = 0.675891
Epoch 167
Validation binary_cross_entropy = 0.718411
Epoch 168
Validation binary_cross_entropy = 0.751128
Epoch 169
Loss = 1.6093e-02, PNorm = 96.5633, GNorm = 0.3598, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.789942
Epoch 170
Validation binary_cross_entropy = 0.796775
Epoch 171
Validation binary_cross_entropy = 0.812028
Epoch 172
Validation binary_cross_entropy = 0.838763
Epoch 173
Validation binary_cross_entropy = 0.848633
Epoch 174
Loss = 3.2434e-02, PNorm = 96.7445, GNorm = 0.5064, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.788197
Epoch 175
Validation binary_cross_entropy = 0.759084
Epoch 176
Validation binary_cross_entropy = 0.771531
Epoch 177
Validation binary_cross_entropy = 0.810442
Epoch 178
Validation binary_cross_entropy = 0.811401
Epoch 179
Loss = 1.5320e-02, PNorm = 96.9121, GNorm = 0.6123, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.811765
Epoch 180
Validation binary_cross_entropy = 0.815199
Epoch 181
Validation binary_cross_entropy = 0.822038
Epoch 182
Validation binary_cross_entropy = 0.858696
Epoch 183
Validation binary_cross_entropy = 0.888102
Epoch 184
Loss = 4.6287e-02, PNorm = 97.0743, GNorm = 0.1552, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.855122
Epoch 185
Validation binary_cross_entropy = 0.837282
Epoch 186
Validation binary_cross_entropy = 0.846272
Epoch 187
Validation binary_cross_entropy = 0.859607
Epoch 188
Validation binary_cross_entropy = 0.876001
Epoch 189
Loss = 1.8458e-02, PNorm = 97.2500, GNorm = 0.5261, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.883980
Epoch 190
Validation binary_cross_entropy = 0.887794
Epoch 191
Validation binary_cross_entropy = 0.889184
Epoch 192
Validation binary_cross_entropy = 0.896417
Epoch 193
Validation binary_cross_entropy = 0.912825
Epoch 194
Loss = 1.8580e-02, PNorm = 97.3988, GNorm = 0.1485, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.932163
Epoch 195
Validation binary_cross_entropy = 0.955684
Epoch 196
Validation binary_cross_entropy = 0.982750
Epoch 197
Validation binary_cross_entropy = 0.996041
Epoch 198
Validation binary_cross_entropy = 0.987716
Epoch 199
Loss = 8.1192e-03, PNorm = 97.5119, GNorm = 0.2539, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.988797
Epoch 200
Validation binary_cross_entropy = 0.998548
Epoch 201
Validation binary_cross_entropy = 1.039313
Epoch 202
Validation binary_cross_entropy = 1.108162
Epoch 203
Validation binary_cross_entropy = 1.096351
Epoch 204
Loss = 1.6718e-02, PNorm = 97.6331, GNorm = 0.2681, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.034412
Epoch 205
Validation binary_cross_entropy = 1.009501
Epoch 206
Validation binary_cross_entropy = 1.001300
Epoch 207
Validation binary_cross_entropy = 1.028393
Epoch 208
Validation binary_cross_entropy = 0.938860
Epoch 209
Loss = 2.4849e-02, PNorm = 97.7900, GNorm = 0.4103, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.840837
Epoch 210
Validation binary_cross_entropy = 0.815591
Epoch 211
Validation binary_cross_entropy = 0.771713
Epoch 212
Validation binary_cross_entropy = 0.765522
Epoch 213
Validation binary_cross_entropy = 0.776094
Epoch 214
Loss = 4.0323e-02, PNorm = 98.0390, GNorm = 1.5003, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.755399
Epoch 215
Validation binary_cross_entropy = 0.706916
Epoch 216
Validation binary_cross_entropy = 0.680955
Epoch 217
Validation binary_cross_entropy = 0.686276
Epoch 218
Validation binary_cross_entropy = 0.707046
Epoch 219
Loss = 2.1039e-02, PNorm = 98.2808, GNorm = 0.2701, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.757028
Epoch 220
Validation binary_cross_entropy = 0.823166
Epoch 221
Validation binary_cross_entropy = 0.845862
Epoch 222
Validation binary_cross_entropy = 0.850736
Epoch 223
Validation binary_cross_entropy = 0.855748
Epoch 224
Loss = 2.7196e-02, PNorm = 98.4562, GNorm = 0.1011, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.881121
Epoch 225
Validation binary_cross_entropy = 0.907843
Epoch 226
Validation binary_cross_entropy = 0.923810
Epoch 227
Validation binary_cross_entropy = 0.909038
Epoch 228
Validation binary_cross_entropy = 0.892218
Epoch 229
Loss = 6.8096e-03, PNorm = 98.5937, GNorm = 0.1223, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.889874
Epoch 230
Validation binary_cross_entropy = 0.916391
Epoch 231
Validation binary_cross_entropy = 0.965680
Epoch 232
Validation binary_cross_entropy = 1.028572
Epoch 233
Validation binary_cross_entropy = 1.021393
Epoch 234
Loss = 2.2750e-02, PNorm = 98.6970, GNorm = 0.7258, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.970359
Epoch 235
Validation binary_cross_entropy = 0.941217
Epoch 236
Validation binary_cross_entropy = 0.930764
Epoch 237
Validation binary_cross_entropy = 0.924684
Epoch 238
Validation binary_cross_entropy = 0.930530
Epoch 239
Loss = 4.4858e-02, PNorm = 98.8082, GNorm = 0.5375, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.960037
Epoch 240
Validation binary_cross_entropy = 0.972970
Epoch 241
Validation binary_cross_entropy = 0.932325
Epoch 242
Validation binary_cross_entropy = 0.899986
Epoch 243
Validation binary_cross_entropy = 0.897518
Epoch 244
Loss = 1.5490e-02, PNorm = 98.9178, GNorm = 0.0309, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.918286
Epoch 245
Validation binary_cross_entropy = 0.942294
Epoch 246
Validation binary_cross_entropy = 0.968290
Epoch 247
Validation binary_cross_entropy = 0.995881
Epoch 248
Validation binary_cross_entropy = 1.013874
Epoch 249
Loss = 1.2389e-02, PNorm = 99.0309, GNorm = 0.3471, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.021533
Epoch 250
Validation binary_cross_entropy = 1.014482
Epoch 251
Validation binary_cross_entropy = 1.002047
Epoch 252
Validation binary_cross_entropy = 1.014596
Epoch 253
Validation binary_cross_entropy = 1.065494
Epoch 254
Loss = 2.3179e-02, PNorm = 99.1266, GNorm = 0.3922, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.089190
Epoch 255
Validation binary_cross_entropy = 1.088195
Epoch 256
Validation binary_cross_entropy = 1.057588
Epoch 257
Validation binary_cross_entropy = 1.031760
Epoch 258
Validation binary_cross_entropy = 1.019012
Epoch 259
Loss = 3.9088e-03, PNorm = 99.2406, GNorm = 0.0031, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.017333
Epoch 260
Validation binary_cross_entropy = 1.051376
Epoch 261
Validation binary_cross_entropy = 1.094284
Epoch 262
Validation binary_cross_entropy = 1.081865
Epoch 263
Validation binary_cross_entropy = 1.027616
Epoch 264
Loss = 2.0369e-02, PNorm = 99.3309, GNorm = 0.2506, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.984444
Epoch 265
Validation binary_cross_entropy = 0.951970
Epoch 266
Validation binary_cross_entropy = 0.945129
Epoch 267
Validation binary_cross_entropy = 0.951605
Epoch 268
Validation binary_cross_entropy = 0.958544
Epoch 269
Loss = 2.6733e-02, PNorm = 99.4565, GNorm = 0.9431, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.976435
Epoch 270
Validation binary_cross_entropy = 0.970790
Epoch 271
Validation binary_cross_entropy = 0.970697
Epoch 272
Validation binary_cross_entropy = 0.986914
Epoch 273
Validation binary_cross_entropy = 1.018056
Epoch 274
Loss = 2.0960e-02, PNorm = 99.5578, GNorm = 0.2603, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.015396
Epoch 275
Validation binary_cross_entropy = 1.016265
Epoch 276
Validation binary_cross_entropy = 0.998857
Epoch 277
Validation binary_cross_entropy = 1.019571
Epoch 278
Validation binary_cross_entropy = 1.007493
Epoch 279
Loss = 2.5614e-02, PNorm = 99.7042, GNorm = 0.8069, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.010098
Epoch 280
Validation binary_cross_entropy = 1.033907
Epoch 281
Validation binary_cross_entropy = 1.039873
Epoch 282
Validation binary_cross_entropy = 1.016674
Epoch 283
Validation binary_cross_entropy = 0.995637
Epoch 284
Loss = 6.1030e-02, PNorm = 99.8874, GNorm = 0.8032, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.020603
Epoch 285
Validation binary_cross_entropy = 1.059265
Epoch 286
Validation binary_cross_entropy = 1.110878
Epoch 287
Validation binary_cross_entropy = 1.146540
Epoch 288
Validation binary_cross_entropy = 1.070617
Epoch 289
Loss = 2.7284e-03, PNorm = 100.0353, GNorm = 0.0200, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.046616
Epoch 290
Validation binary_cross_entropy = 1.052730
Epoch 291
Validation binary_cross_entropy = 1.053896
Epoch 292
Validation binary_cross_entropy = 1.062185
Epoch 293
Validation binary_cross_entropy = 1.079713
Epoch 294
Loss = 1.3022e-02, PNorm = 100.1535, GNorm = 0.3051, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.104660
Epoch 295
Validation binary_cross_entropy = 1.118537
Epoch 296
Validation binary_cross_entropy = 1.136574
Epoch 297
Validation binary_cross_entropy = 1.143762
Epoch 298
Validation binary_cross_entropy = 1.154473
Epoch 299
Loss = 2.8398e-03, PNorm = 100.2595, GNorm = 0.0953, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.157465
Model 0 best validation binary_cross_entropy = 0.302955 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.237489
Ensemble test binary_cross_entropy = 0.237489
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.394170
Epoch 1
Validation binary_cross_entropy = 0.699988
Epoch 2
Validation binary_cross_entropy = 0.260452
Epoch 3
Validation binary_cross_entropy = 0.653033
Epoch 4
Loss = 6.3035e-01, PNorm = 87.7925, GNorm = 3.2407, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.321879
Epoch 5
Validation binary_cross_entropy = 0.391872
Epoch 6
Validation binary_cross_entropy = 0.381516
Epoch 7
Validation binary_cross_entropy = 0.319582
Epoch 8
Validation binary_cross_entropy = 0.412702
Epoch 9
Loss = 3.8814e-01, PNorm = 88.2103, GNorm = 2.0024, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.389671
Epoch 10
Validation binary_cross_entropy = 0.321917
Epoch 11
Validation binary_cross_entropy = 0.347828
Epoch 12
Validation binary_cross_entropy = 0.329249
Epoch 13
Validation binary_cross_entropy = 0.363755
Epoch 14
Loss = 4.9555e-01, PNorm = 88.6197, GNorm = 2.1448, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.385227
Epoch 15
Validation binary_cross_entropy = 0.362096
Epoch 16
Validation binary_cross_entropy = 0.427085
Epoch 17
Validation binary_cross_entropy = 0.375969
Epoch 18
Validation binary_cross_entropy = 0.357120
Epoch 19
Loss = 3.8349e-01, PNorm = 88.9259, GNorm = 1.4967, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.364224
Epoch 20
Validation binary_cross_entropy = 0.374794
Epoch 21
Validation binary_cross_entropy = 0.335709
Epoch 22
Validation binary_cross_entropy = 0.349354
Epoch 23
Validation binary_cross_entropy = 0.324222
Epoch 24
Loss = 2.3992e-01, PNorm = 89.1816, GNorm = 1.9283, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.339618
Epoch 25
Validation binary_cross_entropy = 0.453686
Epoch 26
Validation binary_cross_entropy = 0.347914
Epoch 27
Validation binary_cross_entropy = 0.345716
Epoch 28
Validation binary_cross_entropy = 0.430201
Epoch 29
Loss = 2.6094e-01, PNorm = 89.4198, GNorm = 1.7363, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.334873
Epoch 30
Validation binary_cross_entropy = 0.333181
Epoch 31
Validation binary_cross_entropy = 0.439480
Epoch 32
Validation binary_cross_entropy = 0.432211
Epoch 33
Validation binary_cross_entropy = 0.342307
Epoch 34
Loss = 1.8957e-01, PNorm = 89.6606, GNorm = 1.7895, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.348632
Epoch 35
Validation binary_cross_entropy = 0.372751
Epoch 36
Validation binary_cross_entropy = 0.376993
Epoch 37
Validation binary_cross_entropy = 0.355868
Epoch 38
Validation binary_cross_entropy = 0.354995
Epoch 39
Loss = 3.1434e-01, PNorm = 89.9107, GNorm = 1.8393, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.353300
Epoch 40
Validation binary_cross_entropy = 0.371126
Epoch 41
Validation binary_cross_entropy = 0.363372
Epoch 42
Validation binary_cross_entropy = 0.359144
Epoch 43
Validation binary_cross_entropy = 0.411858
Epoch 44
Loss = 1.3276e-01, PNorm = 90.1817, GNorm = 2.0984, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.446695
Epoch 45
Validation binary_cross_entropy = 0.390619
Epoch 46
Validation binary_cross_entropy = 0.395069
Epoch 47
Validation binary_cross_entropy = 0.409466
Epoch 48
Validation binary_cross_entropy = 0.393413
Epoch 49
Loss = 2.2927e-01, PNorm = 90.4645, GNorm = 3.2966, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.392548
Epoch 50
Validation binary_cross_entropy = 0.376923
Epoch 51
Validation binary_cross_entropy = 0.377967
Epoch 52
Validation binary_cross_entropy = 0.356871
Epoch 53
Validation binary_cross_entropy = 0.354947
Epoch 54
Loss = 1.9655e-01, PNorm = 90.7733, GNorm = 0.9725, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.389370
Epoch 55
Validation binary_cross_entropy = 0.391368
Epoch 56
Validation binary_cross_entropy = 0.367079
Epoch 57
Validation binary_cross_entropy = 0.386016
Epoch 58
Validation binary_cross_entropy = 0.378411
Epoch 59
Loss = 1.7578e-01, PNorm = 91.0734, GNorm = 2.1381, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.413879
Epoch 60
Validation binary_cross_entropy = 0.420463
Epoch 61
Validation binary_cross_entropy = 0.423065
Epoch 62
Validation binary_cross_entropy = 0.447497
Epoch 63
Validation binary_cross_entropy = 0.424090
Epoch 64
Loss = 2.1232e-01, PNorm = 91.3551, GNorm = 2.0860, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.438108
Epoch 65
Validation binary_cross_entropy = 0.399969
Epoch 66
Validation binary_cross_entropy = 0.401772
Epoch 67
Validation binary_cross_entropy = 0.399642
Epoch 68
Validation binary_cross_entropy = 0.399366
Epoch 69
Loss = 1.3635e-01, PNorm = 91.6081, GNorm = 1.8983, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.422531
Epoch 70
Validation binary_cross_entropy = 0.415862
Epoch 71
Validation binary_cross_entropy = 0.431690
Epoch 72
Validation binary_cross_entropy = 0.453897
Epoch 73
Validation binary_cross_entropy = 0.423006
Epoch 74
Loss = 6.8478e-02, PNorm = 91.8574, GNorm = 0.6158, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.409356
Epoch 75
Validation binary_cross_entropy = 0.425450
Epoch 76
Validation binary_cross_entropy = 0.425825
Epoch 77
Validation binary_cross_entropy = 0.463706
Epoch 78
Validation binary_cross_entropy = 0.457878
Epoch 79
Loss = 6.8296e-02, PNorm = 92.0980, GNorm = 0.7310, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.449812
Epoch 80
Validation binary_cross_entropy = 0.461088
Epoch 81
Validation binary_cross_entropy = 0.454690
Epoch 82
Validation binary_cross_entropy = 0.453548
Epoch 83
Validation binary_cross_entropy = 0.462539
Epoch 84
Loss = 1.3659e-01, PNorm = 92.3194, GNorm = 0.9001, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.460395
Epoch 85
Validation binary_cross_entropy = 0.469114
Epoch 86
Validation binary_cross_entropy = 0.468530
Epoch 87
Validation binary_cross_entropy = 0.465359
Epoch 88
Validation binary_cross_entropy = 0.484509
Epoch 89
Loss = 8.2765e-02, PNorm = 92.5644, GNorm = 1.0986, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.487990
Epoch 90
Validation binary_cross_entropy = 0.485322
Epoch 91
Validation binary_cross_entropy = 0.496778
Epoch 92
Validation binary_cross_entropy = 0.506638
Epoch 93
Validation binary_cross_entropy = 0.530137
Epoch 94
Loss = 8.7907e-02, PNorm = 92.8221, GNorm = 1.2333, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.531562
Epoch 95
Validation binary_cross_entropy = 0.504588
Epoch 96
Validation binary_cross_entropy = 0.493763
Epoch 97
Validation binary_cross_entropy = 0.499987
Epoch 98
Validation binary_cross_entropy = 0.516636
Epoch 99
Loss = 4.8326e-02, PNorm = 93.0517, GNorm = 0.5247, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.544759
Epoch 100
Validation binary_cross_entropy = 0.542126
Epoch 101
Validation binary_cross_entropy = 0.563879
Epoch 102
Validation binary_cross_entropy = 0.571503
Epoch 103
Validation binary_cross_entropy = 0.580653
Epoch 104
Loss = 3.5826e-02, PNorm = 93.3462, GNorm = 0.6480, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.569002
Epoch 105
Validation binary_cross_entropy = 0.575087
Epoch 106
Validation binary_cross_entropy = 0.579938
Epoch 107
Validation binary_cross_entropy = 0.597800
Epoch 108
Validation binary_cross_entropy = 0.616734
Epoch 109
Loss = 6.8372e-02, PNorm = 93.6681, GNorm = 0.4274, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.612808
Epoch 110
Validation binary_cross_entropy = 0.634426
Epoch 111
Validation binary_cross_entropy = 0.659690
Epoch 112
Validation binary_cross_entropy = 0.659059
Epoch 113
Validation binary_cross_entropy = 0.629748
Epoch 114
Loss = 2.3886e-02, PNorm = 93.9813, GNorm = 0.3886, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.616529
Epoch 115
Validation binary_cross_entropy = 0.633312
Epoch 116
Validation binary_cross_entropy = 0.617620
Epoch 117
Validation binary_cross_entropy = 0.598554
Epoch 118
Validation binary_cross_entropy = 0.614677
Epoch 119
Loss = 5.6183e-02, PNorm = 94.2859, GNorm = 1.2165, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.617122
Epoch 120
Validation binary_cross_entropy = 0.679676
Epoch 121
Validation binary_cross_entropy = 0.776511
Epoch 122
Validation binary_cross_entropy = 0.825464
Epoch 123
Validation binary_cross_entropy = 0.756388
Epoch 124
Loss = 1.2443e-01, PNorm = 94.6028, GNorm = 1.8428, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.730522
Epoch 125
Validation binary_cross_entropy = 0.667732
Epoch 126
Validation binary_cross_entropy = 0.609297
Epoch 127
Validation binary_cross_entropy = 0.595863
Epoch 128
Validation binary_cross_entropy = 0.616344
Epoch 129
Loss = 6.1881e-02, PNorm = 94.8877, GNorm = 1.5364, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.654092
Epoch 130
Validation binary_cross_entropy = 0.636403
Epoch 131
Validation binary_cross_entropy = 0.626865
Epoch 132
Validation binary_cross_entropy = 0.642557
Epoch 133
Validation binary_cross_entropy = 0.683482
Epoch 134
Loss = 4.4442e-02, PNorm = 95.1814, GNorm = 0.6564, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.686633
Epoch 135
Validation binary_cross_entropy = 0.693500
Epoch 136
Validation binary_cross_entropy = 0.711835
Epoch 137
Validation binary_cross_entropy = 0.729362
Epoch 138
Validation binary_cross_entropy = 0.748913
Epoch 139
Loss = 1.2924e-01, PNorm = 95.4038, GNorm = 1.4570, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.739823
Epoch 140
Validation binary_cross_entropy = 0.701869
Epoch 141
Validation binary_cross_entropy = 0.677670
Epoch 142
Validation binary_cross_entropy = 0.686310
Epoch 143
Validation binary_cross_entropy = 0.713114
Epoch 144
Loss = 5.5702e-02, PNorm = 95.6398, GNorm = 1.0206, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.711074
Epoch 145
Validation binary_cross_entropy = 0.722472
Epoch 146
Validation binary_cross_entropy = 0.764701
Epoch 147
Validation binary_cross_entropy = 0.826759
Epoch 148
Validation binary_cross_entropy = 0.810496
Epoch 149
Loss = 3.7779e-02, PNorm = 95.8861, GNorm = 0.7381, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.810794
Epoch 150
Validation binary_cross_entropy = 0.835033
Epoch 151
Validation binary_cross_entropy = 0.890173
Epoch 152
Validation binary_cross_entropy = 0.856259
Epoch 153
Validation binary_cross_entropy = 0.832511
Epoch 154
Loss = 6.6290e-02, PNorm = 96.0962, GNorm = 0.2783, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.821719
Epoch 155
Validation binary_cross_entropy = 0.806473
Epoch 156
Validation binary_cross_entropy = 0.784382
Epoch 157
Validation binary_cross_entropy = 0.792163
Epoch 158
Validation binary_cross_entropy = 0.792556
Epoch 159
Loss = 2.6979e-02, PNorm = 96.3154, GNorm = 0.3410, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.782386
Epoch 160
Validation binary_cross_entropy = 0.820705
Epoch 161
Validation binary_cross_entropy = 0.789376
Epoch 162
Validation binary_cross_entropy = 0.723142
Epoch 163
Validation binary_cross_entropy = 0.709207
Epoch 164
Loss = 4.3775e-02, PNorm = 96.5487, GNorm = 0.9151, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.722872
Epoch 165
Validation binary_cross_entropy = 0.738603
Epoch 166
Validation binary_cross_entropy = 0.777673
Epoch 167
Validation binary_cross_entropy = 0.771196
Epoch 168
Validation binary_cross_entropy = 0.777896
Epoch 169
Loss = 6.2528e-02, PNorm = 96.8038, GNorm = 1.7510, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.782357
Epoch 170
Validation binary_cross_entropy = 0.754758
Epoch 171
Validation binary_cross_entropy = 0.750893
Epoch 172
Validation binary_cross_entropy = 0.756189
Epoch 173
Validation binary_cross_entropy = 0.782519
Epoch 174
Loss = 1.2962e-01, PNorm = 97.0269, GNorm = 2.8293, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.799730
Epoch 175
Validation binary_cross_entropy = 0.850507
Epoch 176
Validation binary_cross_entropy = 0.873797
Epoch 177
Validation binary_cross_entropy = 0.910001
Epoch 178
Validation binary_cross_entropy = 0.949554
Epoch 179
Loss = 6.7167e-02, PNorm = 97.3446, GNorm = 1.4266, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.878727
Epoch 180
Validation binary_cross_entropy = 0.830236
Epoch 181
Validation binary_cross_entropy = 0.802119
Epoch 182
Validation binary_cross_entropy = 0.774853
Epoch 183
Validation binary_cross_entropy = 0.786693
Epoch 184
Loss = 6.4984e-02, PNorm = 97.6526, GNorm = 1.8242, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.778555
Epoch 185
Validation binary_cross_entropy = 0.746379
Epoch 186
Validation binary_cross_entropy = 0.745903
Epoch 187
Validation binary_cross_entropy = 0.777446
Epoch 188
Validation binary_cross_entropy = 0.789191
Epoch 189
Loss = 4.5086e-02, PNorm = 97.9120, GNorm = 0.9092, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.790390
Epoch 190
Validation binary_cross_entropy = 0.806099
Epoch 191
Validation binary_cross_entropy = 0.800202
Epoch 192
Validation binary_cross_entropy = 0.812959
Epoch 193
Validation binary_cross_entropy = 0.876509
Epoch 194
Loss = 1.9953e-02, PNorm = 98.1321, GNorm = 0.2660, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.939360
Epoch 195
Validation binary_cross_entropy = 0.917821
Epoch 196
Validation binary_cross_entropy = 0.889629
Epoch 197
Validation binary_cross_entropy = 0.889752
Epoch 198
Validation binary_cross_entropy = 0.890094
Epoch 199
Loss = 1.2008e-02, PNorm = 98.3372, GNorm = 0.1468, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.897686
Epoch 200
Validation binary_cross_entropy = 0.915043
Epoch 201
Validation binary_cross_entropy = 0.902233
Epoch 202
Validation binary_cross_entropy = 0.893372
Epoch 203
Validation binary_cross_entropy = 0.891611
Epoch 204
Loss = 2.5615e-02, PNorm = 98.5159, GNorm = 0.4947, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.913147
Epoch 205
Validation binary_cross_entropy = 0.941515
Epoch 206
Validation binary_cross_entropy = 0.946354
Epoch 207
Validation binary_cross_entropy = 0.917117
Epoch 208
Validation binary_cross_entropy = 0.904738
Epoch 209
Loss = 8.4494e-03, PNorm = 98.6341, GNorm = 0.0621, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.901741
Epoch 210
Validation binary_cross_entropy = 0.912912
Epoch 211
Validation binary_cross_entropy = 0.935431
Epoch 212
Validation binary_cross_entropy = 0.953163
Epoch 213
Validation binary_cross_entropy = 0.969389
Epoch 214
Loss = 4.5539e-02, PNorm = 98.7439, GNorm = 1.1454, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.017553
Epoch 215
Validation binary_cross_entropy = 1.047176
Epoch 216
Validation binary_cross_entropy = 1.045488
Epoch 217
Validation binary_cross_entropy = 1.043788
Epoch 218
Validation binary_cross_entropy = 1.029786
Epoch 219
Loss = 1.1580e-02, PNorm = 98.8542, GNorm = 0.1851, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.993646
Epoch 220
Validation binary_cross_entropy = 0.969827
Epoch 221
Validation binary_cross_entropy = 0.968356
Epoch 222
Validation binary_cross_entropy = 1.000003
Epoch 223
Validation binary_cross_entropy = 1.055030
Epoch 224
Loss = 1.5844e-02, PNorm = 98.9612, GNorm = 0.2570, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.054895
Epoch 225
Validation binary_cross_entropy = 1.002066
Epoch 226
Validation binary_cross_entropy = 0.957139
Epoch 227
Validation binary_cross_entropy = 0.927556
Epoch 228
Validation binary_cross_entropy = 0.930116
Epoch 229
Loss = 8.0156e-03, PNorm = 99.0651, GNorm = 0.3217, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.952246
Epoch 230
Validation binary_cross_entropy = 0.980968
Epoch 231
Validation binary_cross_entropy = 0.986551
Epoch 232
Validation binary_cross_entropy = 0.978424
Epoch 233
Validation binary_cross_entropy = 0.972900
Epoch 234
Loss = 6.2585e-03, PNorm = 99.1848, GNorm = 0.2224, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.975230
Epoch 235
Validation binary_cross_entropy = 0.981421
Epoch 236
Validation binary_cross_entropy = 0.992566
Epoch 237
Validation binary_cross_entropy = 1.004073
Epoch 238
Validation binary_cross_entropy = 1.007925
Epoch 239
Loss = 1.6380e-02, PNorm = 99.2623, GNorm = 0.7461, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.021642
Epoch 240
Validation binary_cross_entropy = 1.040593
Epoch 241
Validation binary_cross_entropy = 1.059102
Epoch 242
Validation binary_cross_entropy = 1.084378
Epoch 243
Validation binary_cross_entropy = 1.083719
Epoch 244
Loss = 1.1117e-02, PNorm = 99.3353, GNorm = 0.0597, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.055623
Epoch 245
Validation binary_cross_entropy = 1.028500
Epoch 246
Validation binary_cross_entropy = 1.016530
Epoch 247
Validation binary_cross_entropy = 1.013977
Epoch 248
Validation binary_cross_entropy = 1.037102
Epoch 249
Loss = 2.1635e-03, PNorm = 99.4055, GNorm = 0.0633, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.064853
Epoch 250
Validation binary_cross_entropy = 1.102355
Epoch 251
Validation binary_cross_entropy = 1.161892
Epoch 252
Validation binary_cross_entropy = 1.171382
Epoch 253
Validation binary_cross_entropy = 1.121484
Epoch 254
Loss = 2.4672e-03, PNorm = 99.4727, GNorm = 0.0386, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.087214
Epoch 255
Validation binary_cross_entropy = 1.087496
Epoch 256
Validation binary_cross_entropy = 1.082392
Epoch 257
Validation binary_cross_entropy = 1.085229
Epoch 258
Validation binary_cross_entropy = 1.091987
Epoch 259
Loss = 1.2532e-02, PNorm = 99.5534, GNorm = 1.0887, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.086620
Epoch 260
Validation binary_cross_entropy = 1.078403
Epoch 261
Validation binary_cross_entropy = 1.095598
Epoch 262
Validation binary_cross_entropy = 1.118706
Epoch 263
Validation binary_cross_entropy = 1.142686
Epoch 264
Loss = 4.0682e-03, PNorm = 99.6436, GNorm = 0.1219, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.155327
Epoch 265
Validation binary_cross_entropy = 1.155832
Epoch 266
Validation binary_cross_entropy = 1.152862
Epoch 267
Validation binary_cross_entropy = 1.157680
Epoch 268
Validation binary_cross_entropy = 1.167295
Epoch 269
Loss = 3.9662e-03, PNorm = 99.7134, GNorm = 0.0387, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.181873
Epoch 270
Validation binary_cross_entropy = 1.195677
Epoch 271
Validation binary_cross_entropy = 1.209631
Epoch 272
Validation binary_cross_entropy = 1.207225
Epoch 273
Validation binary_cross_entropy = 1.198694
Epoch 274
Loss = 2.1371e-03, PNorm = 99.7519, GNorm = 0.0556, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.187263
Epoch 275
Validation binary_cross_entropy = 1.178427
Epoch 276
Validation binary_cross_entropy = 1.212891
Epoch 277
Validation binary_cross_entropy = 1.231737
Epoch 278
Validation binary_cross_entropy = 1.245824
Epoch 279
Loss = 1.4147e-02, PNorm = 99.8042, GNorm = 0.1993, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.275651
Epoch 280
Validation binary_cross_entropy = 1.287483
Epoch 281
Validation binary_cross_entropy = 1.296784
Epoch 282
Validation binary_cross_entropy = 1.286172
Epoch 283
Validation binary_cross_entropy = 1.263073
Epoch 284
Loss = 4.3531e-03, PNorm = 99.8786, GNorm = 0.1705, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.235800
Epoch 285
Validation binary_cross_entropy = 1.213331
Epoch 286
Validation binary_cross_entropy = 1.194818
Epoch 287
Validation binary_cross_entropy = 1.177093
Epoch 288
Validation binary_cross_entropy = 1.169646
Epoch 289
Loss = 1.2301e-02, PNorm = 99.9356, GNorm = 0.6547, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.171378
Epoch 290
Validation binary_cross_entropy = 1.203977
Epoch 291
Validation binary_cross_entropy = 1.243146
Epoch 292
Validation binary_cross_entropy = 1.276982
Epoch 293
Validation binary_cross_entropy = 1.344937
Epoch 294
Loss = 2.4661e-02, PNorm = 100.0163, GNorm = 0.9094, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.331815
Epoch 295
Validation binary_cross_entropy = 1.266818
Epoch 296
Validation binary_cross_entropy = 1.220207
Epoch 297
Validation binary_cross_entropy = 1.203695
Epoch 298
Validation binary_cross_entropy = 1.201677
Epoch 299
Loss = 2.9374e-02, PNorm = 100.1145, GNorm = 1.1639, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.198279
Model 0 best validation binary_cross_entropy = 0.260452 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.184349
Ensemble test binary_cross_entropy = 0.184349
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.410761
Epoch 1
Validation binary_cross_entropy = 0.559269
Epoch 2
Validation binary_cross_entropy = 0.562108
Epoch 3
Validation binary_cross_entropy = 0.283851
Epoch 4
Loss = 6.7072e-01, PNorm = 87.7911, GNorm = 2.3419, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.319360
Epoch 5
Validation binary_cross_entropy = 0.473988
Epoch 6
Validation binary_cross_entropy = 0.385969
Epoch 7
Validation binary_cross_entropy = 0.307420
Epoch 8
Validation binary_cross_entropy = 0.467052
Epoch 9
Loss = 4.3973e-01, PNorm = 88.1963, GNorm = 3.6654, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.496477
Epoch 10
Validation binary_cross_entropy = 0.289611
Epoch 11
Validation binary_cross_entropy = 0.346098
Epoch 12
Validation binary_cross_entropy = 0.519042
Epoch 13
Validation binary_cross_entropy = 0.358715
Epoch 14
Loss = 3.0633e-01, PNorm = 88.5988, GNorm = 1.7229, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.284050
Epoch 15
Validation binary_cross_entropy = 0.319820
Epoch 16
Validation binary_cross_entropy = 0.416564
Epoch 17
Validation binary_cross_entropy = 0.350185
Epoch 18
Validation binary_cross_entropy = 0.305422
Epoch 19
Loss = 3.0599e-01, PNorm = 88.8760, GNorm = 2.3794, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.327186
Epoch 20
Validation binary_cross_entropy = 0.387519
Epoch 21
Validation binary_cross_entropy = 0.396424
Epoch 22
Validation binary_cross_entropy = 0.322192
Epoch 23
Validation binary_cross_entropy = 0.315243
Epoch 24
Loss = 2.0852e-01, PNorm = 89.0803, GNorm = 1.1707, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.376830
Epoch 25
Validation binary_cross_entropy = 0.501840
Epoch 26
Validation binary_cross_entropy = 0.384413
Epoch 27
Validation binary_cross_entropy = 0.337653
Epoch 28
Validation binary_cross_entropy = 0.337598
Epoch 29
Loss = 3.1962e-01, PNorm = 89.2636, GNorm = 2.2204, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.351914
Epoch 30
Validation binary_cross_entropy = 0.419169
Epoch 31
Validation binary_cross_entropy = 0.421029
Epoch 32
Validation binary_cross_entropy = 0.371159
Epoch 33
Validation binary_cross_entropy = 0.370723
Epoch 34
Loss = 1.8513e-01, PNorm = 89.4668, GNorm = 1.2359, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.423546
Epoch 35
Validation binary_cross_entropy = 0.418525
Epoch 36
Validation binary_cross_entropy = 0.341084
Epoch 37
Validation binary_cross_entropy = 0.329693
Epoch 38
Validation binary_cross_entropy = 0.347385
Epoch 39
Loss = 2.1616e-01, PNorm = 89.6681, GNorm = 1.5599, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.340018
Epoch 40
Validation binary_cross_entropy = 0.335016
Epoch 41
Validation binary_cross_entropy = 0.346505
Epoch 42
Validation binary_cross_entropy = 0.374680
Epoch 43
Validation binary_cross_entropy = 0.377225
Epoch 44
Loss = 2.5217e-01, PNorm = 89.8934, GNorm = 2.1424, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.326556
Epoch 45
Validation binary_cross_entropy = 0.328932
Epoch 46
Validation binary_cross_entropy = 0.332119
Epoch 47
Validation binary_cross_entropy = 0.376417
Epoch 48
Validation binary_cross_entropy = 0.362610
Epoch 49
Loss = 9.7195e-02, PNorm = 90.1023, GNorm = 1.2798, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.337661
Epoch 50
Validation binary_cross_entropy = 0.348545
Epoch 51
Validation binary_cross_entropy = 0.347591
Epoch 52
Validation binary_cross_entropy = 0.375933
Epoch 53
Validation binary_cross_entropy = 0.382528
Epoch 54
Loss = 2.0989e-01, PNorm = 90.3335, GNorm = 0.9924, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.356470
Epoch 55
Validation binary_cross_entropy = 0.344135
Epoch 56
Validation binary_cross_entropy = 0.340242
Epoch 57
Validation binary_cross_entropy = 0.340357
Epoch 58
Validation binary_cross_entropy = 0.346289
Epoch 59
Loss = 2.1025e-01, PNorm = 90.5561, GNorm = 1.8200, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.365821
Epoch 60
Validation binary_cross_entropy = 0.386879
Epoch 61
Validation binary_cross_entropy = 0.389846
Epoch 62
Validation binary_cross_entropy = 0.416873
Epoch 63
Validation binary_cross_entropy = 0.402805
Epoch 64
Loss = 1.1003e-01, PNorm = 90.7844, GNorm = 0.8025, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.385271
Epoch 65
Validation binary_cross_entropy = 0.388329
Epoch 66
Validation binary_cross_entropy = 0.396812
Epoch 67
Validation binary_cross_entropy = 0.387429
Epoch 68
Validation binary_cross_entropy = 0.392254
Epoch 69
Loss = 1.6359e-01, PNorm = 91.0443, GNorm = 1.9573, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.391998
Epoch 70
Validation binary_cross_entropy = 0.383635
Epoch 71
Validation binary_cross_entropy = 0.379508
Epoch 72
Validation binary_cross_entropy = 0.374287
Epoch 73
Validation binary_cross_entropy = 0.381366
Epoch 74
Loss = 2.0463e-01, PNorm = 91.3095, GNorm = 1.1811, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.399226
Epoch 75
Validation binary_cross_entropy = 0.400131
Epoch 76
Validation binary_cross_entropy = 0.413355
Epoch 77
Validation binary_cross_entropy = 0.431823
Epoch 78
Validation binary_cross_entropy = 0.445945
Epoch 79
Loss = 1.4059e-01, PNorm = 91.5922, GNorm = 2.5320, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.446197
Epoch 80
Validation binary_cross_entropy = 0.435100
Epoch 81
Validation binary_cross_entropy = 0.448536
Epoch 82
Validation binary_cross_entropy = 0.448713
Epoch 83
Validation binary_cross_entropy = 0.443745
Epoch 84
Loss = 1.1008e-01, PNorm = 91.8999, GNorm = 0.8857, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.444362
Epoch 85
Validation binary_cross_entropy = 0.453878
Epoch 86
Validation binary_cross_entropy = 0.454267
Epoch 87
Validation binary_cross_entropy = 0.461357
Epoch 88
Validation binary_cross_entropy = 0.483028
Epoch 89
Loss = 6.5414e-02, PNorm = 92.2118, GNorm = 0.6874, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.501575
Epoch 90
Validation binary_cross_entropy = 0.502867
Epoch 91
Validation binary_cross_entropy = 0.507189
Epoch 92
Validation binary_cross_entropy = 0.519727
Epoch 93
Validation binary_cross_entropy = 0.527553
Epoch 94
Loss = 5.5674e-02, PNorm = 92.4817, GNorm = 0.8918, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.526703
Epoch 95
Validation binary_cross_entropy = 0.520376
Epoch 96
Validation binary_cross_entropy = 0.516544
Epoch 97
Validation binary_cross_entropy = 0.505603
Epoch 98
Validation binary_cross_entropy = 0.477726
Epoch 99
Loss = 7.7339e-02, PNorm = 92.7232, GNorm = 0.9958, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.477436
Epoch 100
Validation binary_cross_entropy = 0.457748
Epoch 101
Validation binary_cross_entropy = 0.472446
Epoch 102
Validation binary_cross_entropy = 0.494736
Epoch 103
Validation binary_cross_entropy = 0.515414
Epoch 104
Loss = 5.0476e-02, PNorm = 92.9859, GNorm = 0.8231, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.531569
Epoch 105
Validation binary_cross_entropy = 0.538580
Epoch 106
Validation binary_cross_entropy = 0.543480
Epoch 107
Validation binary_cross_entropy = 0.546413
Epoch 108
Validation binary_cross_entropy = 0.542879
Epoch 109
Loss = 4.4257e-02, PNorm = 93.2119, GNorm = 0.8063, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.517613
Epoch 110
Validation binary_cross_entropy = 0.519703
Epoch 111
Validation binary_cross_entropy = 0.523113
Epoch 112
Validation binary_cross_entropy = 0.560366
Epoch 113
Validation binary_cross_entropy = 0.556893
Epoch 114
Loss = 5.3507e-02, PNorm = 93.4215, GNorm = 0.5921, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.547156
Epoch 115
Validation binary_cross_entropy = 0.545992
Epoch 116
Validation binary_cross_entropy = 0.578921
Epoch 117
Validation binary_cross_entropy = 0.614991
Epoch 118
Validation binary_cross_entropy = 0.616632
Epoch 119
Loss = 3.7158e-02, PNorm = 93.6981, GNorm = 0.6844, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.604365
Epoch 120
Validation binary_cross_entropy = 0.582756
Epoch 121
Validation binary_cross_entropy = 0.588752
Epoch 122
Validation binary_cross_entropy = 0.618693
Epoch 123
Validation binary_cross_entropy = 0.606324
Epoch 124
Loss = 5.6202e-02, PNorm = 93.9737, GNorm = 0.6991, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.597825
Epoch 125
Validation binary_cross_entropy = 0.601318
Epoch 126
Validation binary_cross_entropy = 0.625680
Epoch 127
Validation binary_cross_entropy = 0.656003
Epoch 128
Validation binary_cross_entropy = 0.623358
Epoch 129
Loss = 4.7261e-02, PNorm = 94.2359, GNorm = 0.4563, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.591834
Epoch 130
Validation binary_cross_entropy = 0.587127
Epoch 131
Validation binary_cross_entropy = 0.639764
Epoch 132
Validation binary_cross_entropy = 0.604694
Epoch 133
Validation binary_cross_entropy = 0.558406
Epoch 134
Loss = 5.6921e-02, PNorm = 94.5153, GNorm = 0.8950, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.548492
Epoch 135
Validation binary_cross_entropy = 0.544690
Epoch 136
Validation binary_cross_entropy = 0.550339
Epoch 137
Validation binary_cross_entropy = 0.556391
Epoch 138
Validation binary_cross_entropy = 0.570675
Epoch 139
Loss = 1.0641e-01, PNorm = 94.7745, GNorm = 1.8615, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.594453
Epoch 140
Validation binary_cross_entropy = 0.605231
Epoch 141
Validation binary_cross_entropy = 0.587050
Epoch 142
Validation binary_cross_entropy = 0.590970
Epoch 143
Validation binary_cross_entropy = 0.607955
Epoch 144
Loss = 2.8114e-02, PNorm = 94.9994, GNorm = 0.2355, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.617037
Epoch 145
Validation binary_cross_entropy = 0.655182
Epoch 146
Validation binary_cross_entropy = 0.685144
Epoch 147
Validation binary_cross_entropy = 0.672304
Epoch 148
Validation binary_cross_entropy = 0.641117
Epoch 149
Loss = 5.5786e-02, PNorm = 95.2181, GNorm = 0.1367, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.644749
Epoch 150
Validation binary_cross_entropy = 0.660084
Epoch 151
Validation binary_cross_entropy = 0.684340
Epoch 152
Validation binary_cross_entropy = 0.688047
Epoch 153
Validation binary_cross_entropy = 0.676483
Epoch 154
Loss = 4.7707e-02, PNorm = 95.4372, GNorm = 0.9271, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.680078
Epoch 155
Validation binary_cross_entropy = 0.681115
Epoch 156
Validation binary_cross_entropy = 0.684288
Epoch 157
Validation binary_cross_entropy = 0.655600
Epoch 158
Validation binary_cross_entropy = 0.693440
Epoch 159
Loss = 2.7396e-02, PNorm = 95.6420, GNorm = 0.4387, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.678715
Epoch 160
Validation binary_cross_entropy = 0.693799
Epoch 161
Validation binary_cross_entropy = 0.763165
Epoch 162
Validation binary_cross_entropy = 0.759624
Epoch 163
Validation binary_cross_entropy = 0.733032
Epoch 164
Loss = 3.0253e-02, PNorm = 95.8812, GNorm = 0.5644, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.746738
Epoch 165
Validation binary_cross_entropy = 0.767099
Epoch 166
Validation binary_cross_entropy = 0.762753
Epoch 167
Validation binary_cross_entropy = 0.702060
Epoch 168
Validation binary_cross_entropy = 0.682743
Epoch 169
Loss = 1.4262e-02, PNorm = 96.1336, GNorm = 0.1458, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.681172
Epoch 170
Validation binary_cross_entropy = 0.725670
Epoch 171
Validation binary_cross_entropy = 0.738779
Epoch 172
Validation binary_cross_entropy = 0.734856
Epoch 173
Validation binary_cross_entropy = 0.660465
Epoch 174
Loss = 6.1622e-02, PNorm = 96.3830, GNorm = 0.2699, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.635064
Epoch 175
Validation binary_cross_entropy = 0.619029
Epoch 176
Validation binary_cross_entropy = 0.634704
Epoch 177
Validation binary_cross_entropy = 0.660434
Epoch 178
Validation binary_cross_entropy = 0.684807
Epoch 179
Loss = 3.4128e-02, PNorm = 96.6316, GNorm = 0.5942, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.713316
Epoch 180
Validation binary_cross_entropy = 0.737804
Epoch 181
Validation binary_cross_entropy = 0.745507
Epoch 182
Validation binary_cross_entropy = 0.755057
Epoch 183
Validation binary_cross_entropy = 0.767471
Epoch 184
Loss = 2.0333e-02, PNorm = 96.8870, GNorm = 0.3816, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.791317
Epoch 185
Validation binary_cross_entropy = 0.802235
Epoch 186
Validation binary_cross_entropy = 0.840317
Epoch 187
Validation binary_cross_entropy = 0.846657
Epoch 188
Validation binary_cross_entropy = 0.773788
Epoch 189
Loss = 1.8918e-02, PNorm = 97.1096, GNorm = 0.4626, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.713676
Epoch 190
Validation binary_cross_entropy = 0.735837
Epoch 191
Validation binary_cross_entropy = 0.802285
Epoch 192
Validation binary_cross_entropy = 0.882724
Epoch 193
Validation binary_cross_entropy = 0.888485
Epoch 194
Loss = 2.3771e-02, PNorm = 97.3551, GNorm = 0.7079, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.845654
Epoch 195
Validation binary_cross_entropy = 0.827407
Epoch 196
Validation binary_cross_entropy = 0.785043
Epoch 197
Validation binary_cross_entropy = 0.728855
Epoch 198
Validation binary_cross_entropy = 0.699701
Epoch 199
Loss = 4.3719e-02, PNorm = 97.5796, GNorm = 1.2957, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.679161
Epoch 200
Validation binary_cross_entropy = 0.688877
Epoch 201
Validation binary_cross_entropy = 0.719710
Epoch 202
Validation binary_cross_entropy = 0.777476
Epoch 203
Validation binary_cross_entropy = 0.774251
Epoch 204
Loss = 4.2565e-02, PNorm = 97.8472, GNorm = 0.9698, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.756994
Epoch 205
Validation binary_cross_entropy = 0.732139
Epoch 206
Validation binary_cross_entropy = 0.732079
Epoch 207
Validation binary_cross_entropy = 0.744317
Epoch 208
Validation binary_cross_entropy = 0.752383
Epoch 209
Loss = 1.7837e-02, PNorm = 98.0536, GNorm = 0.3822, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.748775
Epoch 210
Validation binary_cross_entropy = 0.747590
Epoch 211
Validation binary_cross_entropy = 0.790499
Epoch 212
Validation binary_cross_entropy = 0.863762
Epoch 213
Validation binary_cross_entropy = 0.933492
Epoch 214
Loss = 6.8281e-02, PNorm = 98.2387, GNorm = 0.4361, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.857346
Epoch 215
Validation binary_cross_entropy = 0.824228
Epoch 216
Validation binary_cross_entropy = 0.815699
Epoch 217
Validation binary_cross_entropy = 0.816114
Epoch 218
Validation binary_cross_entropy = 0.809729
Epoch 219
Loss = 1.6205e-02, PNorm = 98.4041, GNorm = 0.2495, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.800050
Epoch 220
Validation binary_cross_entropy = 0.817822
Epoch 221
Validation binary_cross_entropy = 0.851872
Epoch 222
Validation binary_cross_entropy = 0.863965
Epoch 223
Validation binary_cross_entropy = 0.859623
Epoch 224
Loss = 7.7145e-02, PNorm = 98.5651, GNorm = 2.6189, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.876751
Epoch 225
Validation binary_cross_entropy = 0.944639
Epoch 226
Validation binary_cross_entropy = 0.926798
Epoch 227
Validation binary_cross_entropy = 0.865952
Epoch 228
Validation binary_cross_entropy = 0.735677
Epoch 229
Loss = 9.4219e-02, PNorm = 98.7702, GNorm = 1.9505, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.739025
Epoch 230
Validation binary_cross_entropy = 0.807309
Epoch 231
Validation binary_cross_entropy = 0.909187
Epoch 232
Validation binary_cross_entropy = 1.002888
Epoch 233
Validation binary_cross_entropy = 1.056171
Epoch 234
Loss = 1.2000e-02, PNorm = 99.0304, GNorm = 0.4703, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.052379
Epoch 235
Validation binary_cross_entropy = 1.022226
Epoch 236
Validation binary_cross_entropy = 0.993630
Epoch 237
Validation binary_cross_entropy = 0.975280
Epoch 238
Validation binary_cross_entropy = 0.966580
Epoch 239
Loss = 2.9682e-02, PNorm = 99.2124, GNorm = 0.8805, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.961474
Epoch 240
Validation binary_cross_entropy = 0.909514
Epoch 241
Validation binary_cross_entropy = 0.867347
Epoch 242
Validation binary_cross_entropy = 0.856713
Epoch 243
Validation binary_cross_entropy = 0.849233
Epoch 244
Loss = 2.8746e-02, PNorm = 99.3519, GNorm = 1.0222, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.842595
Epoch 245
Validation binary_cross_entropy = 0.854383
Epoch 246
Validation binary_cross_entropy = 0.882613
Epoch 247
Validation binary_cross_entropy = 0.890661
Epoch 248
Validation binary_cross_entropy = 0.895621
Epoch 249
Loss = 4.2792e-03, PNorm = 99.4800, GNorm = 0.1309, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.896451
Epoch 250
Validation binary_cross_entropy = 0.911401
Epoch 251
Validation binary_cross_entropy = 0.919302
Epoch 252
Validation binary_cross_entropy = 0.924814
Epoch 253
Validation binary_cross_entropy = 0.931319
Epoch 254
Loss = 1.1812e-02, PNorm = 99.5985, GNorm = 0.0952, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.938912
Epoch 255
Validation binary_cross_entropy = 0.921798
Epoch 256
Validation binary_cross_entropy = 0.903757
Epoch 257
Validation binary_cross_entropy = 0.894342
Epoch 258
Validation binary_cross_entropy = 0.891437
Epoch 259
Loss = 1.4165e-02, PNorm = 99.6856, GNorm = 0.3012, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.891510
Epoch 260
Validation binary_cross_entropy = 0.883804
Epoch 261
Validation binary_cross_entropy = 0.876777
Epoch 262
Validation binary_cross_entropy = 0.880725
Epoch 263
Validation binary_cross_entropy = 0.871742
Epoch 264
Loss = 3.0575e-02, PNorm = 99.7647, GNorm = 0.8024, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.872227
Epoch 265
Validation binary_cross_entropy = 0.883705
Epoch 266
Validation binary_cross_entropy = 0.892224
Epoch 267
Validation binary_cross_entropy = 0.899872
Epoch 268
Validation binary_cross_entropy = 0.903582
Epoch 269
Loss = 7.0116e-03, PNorm = 99.8459, GNorm = 0.0162, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.915570
Epoch 270
Validation binary_cross_entropy = 0.958801
Epoch 271
Validation binary_cross_entropy = 0.983888
Epoch 272
Validation binary_cross_entropy = 0.972742
Epoch 273
Validation binary_cross_entropy = 0.956107
Epoch 274
Loss = 3.7467e-03, PNorm = 99.9461, GNorm = 0.0635, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.936715
Epoch 275
Validation binary_cross_entropy = 0.931832
Epoch 276
Validation binary_cross_entropy = 0.954447
Epoch 277
Validation binary_cross_entropy = 1.005520
Epoch 278
Validation binary_cross_entropy = 1.030777
Epoch 279
Loss = 1.0887e-02, PNorm = 100.0258, GNorm = 0.3324, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.013041
Epoch 280
Validation binary_cross_entropy = 0.963305
Epoch 281
Validation binary_cross_entropy = 0.937163
Epoch 282
Validation binary_cross_entropy = 0.915176
Epoch 283
Validation binary_cross_entropy = 0.902983
Epoch 284
Loss = 2.8377e-02, PNorm = 100.1225, GNorm = 1.3045, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.907271
Epoch 285
Validation binary_cross_entropy = 0.916741
Epoch 286
Validation binary_cross_entropy = 0.929862
Epoch 287
Validation binary_cross_entropy = 0.938581
Epoch 288
Validation binary_cross_entropy = 0.941697
Epoch 289
Loss = 2.1147e-02, PNorm = 100.1878, GNorm = 0.9178, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.949207
Epoch 290
Validation binary_cross_entropy = 0.965612
Epoch 291
Validation binary_cross_entropy = 0.966592
Epoch 292
Validation binary_cross_entropy = 0.967408
Epoch 293
Validation binary_cross_entropy = 0.996769
Epoch 294
Loss = 2.6827e-02, PNorm = 100.2661, GNorm = 0.0383, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.034263
Epoch 295
Validation binary_cross_entropy = 1.042192
Epoch 296
Validation binary_cross_entropy = 1.046821
Epoch 297
Validation binary_cross_entropy = 1.069764
Epoch 298
Validation binary_cross_entropy = 1.107011
Epoch 299
Loss = 1.7950e-02, PNorm = 100.3632, GNorm = 0.1493, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.177514
Model 0 best validation binary_cross_entropy = 0.283851 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.244032
Ensemble test binary_cross_entropy = 0.244032
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.536160
Epoch 1
Validation binary_cross_entropy = 0.393324
Epoch 2
Validation binary_cross_entropy = 0.247829
Epoch 3
Validation binary_cross_entropy = 0.811346
Epoch 4
Loss = 4.7560e-01, PNorm = 87.8005, GNorm = 2.1434, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.268168
Epoch 5
Validation binary_cross_entropy = 0.527315
Epoch 6
Validation binary_cross_entropy = 0.507664
Epoch 7
Validation binary_cross_entropy = 0.334597
Epoch 8
Validation binary_cross_entropy = 0.399955
Epoch 9
Loss = 3.6872e-01, PNorm = 88.2129, GNorm = 2.2511, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.461362
Epoch 10
Validation binary_cross_entropy = 0.333053
Epoch 11
Validation binary_cross_entropy = 0.385547
Epoch 12
Validation binary_cross_entropy = 0.419256
Epoch 13
Validation binary_cross_entropy = 0.319790
Epoch 14
Loss = 4.6420e-01, PNorm = 88.6120, GNorm = 3.0145, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.325314
Epoch 15
Validation binary_cross_entropy = 0.423559
Epoch 16
Validation binary_cross_entropy = 0.400077
Epoch 17
Validation binary_cross_entropy = 0.331510
Epoch 18
Validation binary_cross_entropy = 0.336040
Epoch 19
Loss = 2.7807e-01, PNorm = 88.8800, GNorm = 1.3172, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.372385
Epoch 20
Validation binary_cross_entropy = 0.385809
Epoch 21
Validation binary_cross_entropy = 0.340170
Epoch 22
Validation binary_cross_entropy = 0.354508
Epoch 23
Validation binary_cross_entropy = 0.500200
Epoch 24
Loss = 3.3377e-01, PNorm = 89.0846, GNorm = 1.6645, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.337292
Epoch 25
Validation binary_cross_entropy = 0.326563
Epoch 26
Validation binary_cross_entropy = 0.355221
Epoch 27
Validation binary_cross_entropy = 0.403605
Epoch 28
Validation binary_cross_entropy = 0.357237
Epoch 29
Loss = 2.6018e-01, PNorm = 89.2747, GNorm = 1.8459, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.367293
Epoch 30
Validation binary_cross_entropy = 0.416382
Epoch 31
Validation binary_cross_entropy = 0.377499
Epoch 32
Validation binary_cross_entropy = 0.377238
Epoch 33
Validation binary_cross_entropy = 0.360662
Epoch 34
Loss = 2.7625e-01, PNorm = 89.5026, GNorm = 2.0868, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.368568
Epoch 35
Validation binary_cross_entropy = 0.393572
Epoch 36
Validation binary_cross_entropy = 0.421072
Epoch 37
Validation binary_cross_entropy = 0.365048
Epoch 38
Validation binary_cross_entropy = 0.358151
Epoch 39
Loss = 2.1313e-01, PNorm = 89.7432, GNorm = 1.6734, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.383328
Epoch 40
Validation binary_cross_entropy = 0.464182
Epoch 41
Validation binary_cross_entropy = 0.364093
Epoch 42
Validation binary_cross_entropy = 0.348368
Epoch 43
Validation binary_cross_entropy = 0.343802
Epoch 44
Loss = 2.1428e-01, PNorm = 90.0044, GNorm = 1.1394, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.338690
Epoch 45
Validation binary_cross_entropy = 0.370689
Epoch 46
Validation binary_cross_entropy = 0.349194
Epoch 47
Validation binary_cross_entropy = 0.364023
Epoch 48
Validation binary_cross_entropy = 0.373458
Epoch 49
Loss = 1.5398e-01, PNorm = 90.2377, GNorm = 1.0506, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.408965
Epoch 50
Validation binary_cross_entropy = 0.416239
Epoch 51
Validation binary_cross_entropy = 0.378034
Epoch 52
Validation binary_cross_entropy = 0.371582
Epoch 53
Validation binary_cross_entropy = 0.371171
Epoch 54
Loss = 1.1309e-01, PNorm = 90.4595, GNorm = 0.8919, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.380048
Epoch 55
Validation binary_cross_entropy = 0.388576
Epoch 56
Validation binary_cross_entropy = 0.377696
Epoch 57
Validation binary_cross_entropy = 0.378287
Epoch 58
Validation binary_cross_entropy = 0.388523
Epoch 59
Loss = 2.2209e-01, PNorm = 90.6996, GNorm = 1.8862, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.411041
Epoch 60
Validation binary_cross_entropy = 0.414884
Epoch 61
Validation binary_cross_entropy = 0.420489
Epoch 62
Validation binary_cross_entropy = 0.415485
Epoch 63
Validation binary_cross_entropy = 0.402985
Epoch 64
Loss = 1.6521e-01, PNorm = 90.9563, GNorm = 1.3040, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.396621
Epoch 65
Validation binary_cross_entropy = 0.393231
Epoch 66
Validation binary_cross_entropy = 0.394763
Epoch 67
Validation binary_cross_entropy = 0.393815
Epoch 68
Validation binary_cross_entropy = 0.422902
Epoch 69
Loss = 2.0489e-01, PNorm = 91.2014, GNorm = 0.8453, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.427469
Epoch 70
Validation binary_cross_entropy = 0.444585
Epoch 71
Validation binary_cross_entropy = 0.452256
Epoch 72
Validation binary_cross_entropy = 0.430657
Epoch 73
Validation binary_cross_entropy = 0.426388
Epoch 74
Loss = 9.1339e-02, PNorm = 91.4755, GNorm = 0.6627, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.426466
Epoch 75
Validation binary_cross_entropy = 0.438661
Epoch 76
Validation binary_cross_entropy = 0.423835
Epoch 77
Validation binary_cross_entropy = 0.420592
Epoch 78
Validation binary_cross_entropy = 0.418510
Epoch 79
Loss = 8.9426e-02, PNorm = 91.7734, GNorm = 1.1121, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.446983
Epoch 80
Validation binary_cross_entropy = 0.429171
Epoch 81
Validation binary_cross_entropy = 0.452754
Epoch 82
Validation binary_cross_entropy = 0.453198
Epoch 83
Validation binary_cross_entropy = 0.454238
Epoch 84
Loss = 8.5036e-02, PNorm = 92.0475, GNorm = 0.9981, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.458423
Epoch 85
Validation binary_cross_entropy = 0.452610
Epoch 86
Validation binary_cross_entropy = 0.471175
Epoch 87
Validation binary_cross_entropy = 0.454951
Epoch 88
Validation binary_cross_entropy = 0.456819
Epoch 89
Loss = 9.6836e-02, PNorm = 92.3265, GNorm = 0.5619, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.470163
Epoch 90
Validation binary_cross_entropy = 0.467415
Epoch 91
Validation binary_cross_entropy = 0.470912
Epoch 92
Validation binary_cross_entropy = 0.472410
Epoch 93
Validation binary_cross_entropy = 0.478872
Epoch 94
Loss = 1.2557e-01, PNorm = 92.5761, GNorm = 1.4244, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.493036
Epoch 95
Validation binary_cross_entropy = 0.514964
Epoch 96
Validation binary_cross_entropy = 0.521179
Epoch 97
Validation binary_cross_entropy = 0.548550
Epoch 98
Validation binary_cross_entropy = 0.566955
Epoch 99
Loss = 4.6064e-02, PNorm = 92.8645, GNorm = 0.6849, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.591582
Epoch 100
Validation binary_cross_entropy = 0.603402
Epoch 101
Validation binary_cross_entropy = 0.545561
Epoch 102
Validation binary_cross_entropy = 0.523649
Epoch 103
Validation binary_cross_entropy = 0.522516
Epoch 104
Loss = 6.1085e-02, PNorm = 93.1205, GNorm = 0.6812, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.522139
Epoch 105
Validation binary_cross_entropy = 0.527412
Epoch 106
Validation binary_cross_entropy = 0.557436
Epoch 107
Validation binary_cross_entropy = 0.587008
Epoch 108
Validation binary_cross_entropy = 0.623274
Epoch 109
Loss = 9.3269e-02, PNorm = 93.4457, GNorm = 0.7812, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.589861
Epoch 110
Validation binary_cross_entropy = 0.561091
Epoch 111
Validation binary_cross_entropy = 0.551776
Epoch 112
Validation binary_cross_entropy = 0.588848
Epoch 113
Validation binary_cross_entropy = 0.576757
Epoch 114
Loss = 7.4017e-02, PNorm = 93.7734, GNorm = 0.5476, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.544178
Epoch 115
Validation binary_cross_entropy = 0.530601
Epoch 116
Validation binary_cross_entropy = 0.523434
Epoch 117
Validation binary_cross_entropy = 0.536765
Epoch 118
Validation binary_cross_entropy = 0.562374
Epoch 119
Loss = 1.7141e-01, PNorm = 94.0926, GNorm = 1.2941, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.616744
Epoch 120
Validation binary_cross_entropy = 0.620550
Epoch 121
Validation binary_cross_entropy = 0.557922
Epoch 122
Validation binary_cross_entropy = 0.581227
Epoch 123
Validation binary_cross_entropy = 0.538561
Epoch 124
Loss = 1.2289e-01, PNorm = 94.4036, GNorm = 1.0789, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.557106
Epoch 125
Validation binary_cross_entropy = 0.558120
Epoch 126
Validation binary_cross_entropy = 0.598878
Epoch 127
Validation binary_cross_entropy = 0.604372
Epoch 128
Validation binary_cross_entropy = 0.609958
Epoch 129
Loss = 5.4722e-02, PNorm = 94.6922, GNorm = 0.7190, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.607475
Epoch 130
Validation binary_cross_entropy = 0.640177
Epoch 131
Validation binary_cross_entropy = 0.656299
Epoch 132
Validation binary_cross_entropy = 0.668591
Epoch 133
Validation binary_cross_entropy = 0.650706
Epoch 134
Loss = 3.0720e-02, PNorm = 94.9321, GNorm = 0.4967, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.631750
Epoch 135
Validation binary_cross_entropy = 0.632569
Epoch 136
Validation binary_cross_entropy = 0.651352
Epoch 137
Validation binary_cross_entropy = 0.683449
Epoch 138
Validation binary_cross_entropy = 0.679759
Epoch 139
Loss = 1.8351e-02, PNorm = 95.1779, GNorm = 0.3002, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.668936
Epoch 140
Validation binary_cross_entropy = 0.723597
Epoch 141
Validation binary_cross_entropy = 0.674967
Epoch 142
Validation binary_cross_entropy = 0.732356
Epoch 143
Validation binary_cross_entropy = 0.730672
Epoch 144
Loss = 3.4776e-02, PNorm = 95.5508, GNorm = 0.7320, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.669872
Epoch 145
Validation binary_cross_entropy = 0.642050
Epoch 146
Validation binary_cross_entropy = 0.638135
Epoch 147
Validation binary_cross_entropy = 0.636030
Epoch 148
Validation binary_cross_entropy = 0.673324
Epoch 149
Loss = 4.6702e-02, PNorm = 95.8563, GNorm = 0.9684, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.678224
Epoch 150
Validation binary_cross_entropy = 0.670912
Epoch 151
Validation binary_cross_entropy = 0.668630
Epoch 152
Validation binary_cross_entropy = 0.679597
Epoch 153
Validation binary_cross_entropy = 0.692355
Epoch 154
Loss = 4.7760e-02, PNorm = 96.0926, GNorm = 1.1882, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.726031
Epoch 155
Validation binary_cross_entropy = 0.757262
Epoch 156
Validation binary_cross_entropy = 0.753671
Epoch 157
Validation binary_cross_entropy = 0.718323
Epoch 158
Validation binary_cross_entropy = 0.697306
Epoch 159
Loss = 3.3032e-02, PNorm = 96.3512, GNorm = 0.6372, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.695613
Epoch 160
Validation binary_cross_entropy = 0.729564
Epoch 161
Validation binary_cross_entropy = 0.772126
Epoch 162
Validation binary_cross_entropy = 0.774130
Epoch 163
Validation binary_cross_entropy = 0.737960
Epoch 164
Loss = 8.5176e-02, PNorm = 96.5837, GNorm = 1.0210, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.691473
Epoch 165
Validation binary_cross_entropy = 0.660182
Epoch 166
Validation binary_cross_entropy = 0.639421
Epoch 167
Validation binary_cross_entropy = 0.627844
Epoch 168
Validation binary_cross_entropy = 0.623950
Epoch 169
Loss = 2.9256e-02, PNorm = 96.8104, GNorm = 0.3978, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.651817
Epoch 170
Validation binary_cross_entropy = 0.689013
Epoch 171
Validation binary_cross_entropy = 0.733164
Epoch 172
Validation binary_cross_entropy = 0.766347
Epoch 173
Validation binary_cross_entropy = 0.826679
Epoch 174
Loss = 1.9472e-02, PNorm = 97.0087, GNorm = 0.6113, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.875261
Epoch 175
Validation binary_cross_entropy = 0.887042
Epoch 176
Validation binary_cross_entropy = 0.879889
Epoch 177
Validation binary_cross_entropy = 0.857367
Epoch 178
Validation binary_cross_entropy = 0.850186
Epoch 179
Loss = 1.8276e-02, PNorm = 97.1343, GNorm = 0.2300, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.837002
Epoch 180
Validation binary_cross_entropy = 0.802343
Epoch 181
Validation binary_cross_entropy = 0.785688
Epoch 182
Validation binary_cross_entropy = 0.761205
Epoch 183
Validation binary_cross_entropy = 0.737991
Epoch 184
Loss = 5.1493e-02, PNorm = 97.2738, GNorm = 2.2542, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.749086
Epoch 185
Validation binary_cross_entropy = 0.784121
Epoch 186
Validation binary_cross_entropy = 0.832260
Epoch 187
Validation binary_cross_entropy = 0.842124
Epoch 188
Validation binary_cross_entropy = 0.804974
Epoch 189
Loss = 3.6054e-02, PNorm = 97.4762, GNorm = 1.5027, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.805112
Epoch 190
Validation binary_cross_entropy = 0.780232
Epoch 191
Validation binary_cross_entropy = 0.745102
Epoch 192
Validation binary_cross_entropy = 0.719622
Epoch 193
Validation binary_cross_entropy = 0.701568
Epoch 194
Loss = 6.6027e-02, PNorm = 97.7183, GNorm = 2.0133, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.680541
Epoch 195
Validation binary_cross_entropy = 0.666981
Epoch 196
Validation binary_cross_entropy = 0.682304
Epoch 197
Validation binary_cross_entropy = 0.710400
Epoch 198
Validation binary_cross_entropy = 0.754450
Epoch 199
Loss = 2.9357e-02, PNorm = 97.9700, GNorm = 1.0083, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.824450
Epoch 200
Validation binary_cross_entropy = 0.847525
Epoch 201
Validation binary_cross_entropy = 0.841504
Epoch 202
Validation binary_cross_entropy = 0.829607
Epoch 203
Validation binary_cross_entropy = 0.814123
Epoch 204
Loss = 7.5520e-02, PNorm = 98.1408, GNorm = 1.7344, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.840986
Epoch 205
Validation binary_cross_entropy = 0.896065
Epoch 206
Validation binary_cross_entropy = 0.947067
Epoch 207
Validation binary_cross_entropy = 0.941458
Epoch 208
Validation binary_cross_entropy = 0.948232
Epoch 209
Loss = 1.5881e-02, PNorm = 98.3195, GNorm = 1.0912, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.909304
Epoch 210
Validation binary_cross_entropy = 0.832521
Epoch 211
Validation binary_cross_entropy = 0.793314
Epoch 212
Validation binary_cross_entropy = 0.775006
Epoch 213
Validation binary_cross_entropy = 0.771633
Epoch 214
Loss = 1.5193e-02, PNorm = 98.4828, GNorm = 0.4831, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.784969
Epoch 215
Validation binary_cross_entropy = 0.779418
Epoch 216
Validation binary_cross_entropy = 0.762303
Epoch 217
Validation binary_cross_entropy = 0.741557
Epoch 218
Validation binary_cross_entropy = 0.742284
Epoch 219
Loss = 1.7389e-02, PNorm = 98.6409, GNorm = 0.4996, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.745897
Epoch 220
Validation binary_cross_entropy = 0.744836
Epoch 221
Validation binary_cross_entropy = 0.744607
Epoch 222
Validation binary_cross_entropy = 0.763275
Epoch 223
Validation binary_cross_entropy = 0.793124
Epoch 224
Loss = 2.6864e-02, PNorm = 98.7852, GNorm = 0.5111, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.796135
Epoch 225
Validation binary_cross_entropy = 0.796898
Epoch 226
Validation binary_cross_entropy = 0.799308
Epoch 227
Validation binary_cross_entropy = 0.810526
Epoch 228
Validation binary_cross_entropy = 0.822017
Epoch 229
Loss = 4.4703e-02, PNorm = 98.9179, GNorm = 0.4573, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.860558
Epoch 230
Validation binary_cross_entropy = 0.901497
Epoch 231
Validation binary_cross_entropy = 0.915394
Epoch 232
Validation binary_cross_entropy = 0.936655
Epoch 233
Validation binary_cross_entropy = 0.948397
Epoch 234
Loss = 3.4764e-03, PNorm = 99.0457, GNorm = 0.0815, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.966325
Epoch 235
Validation binary_cross_entropy = 0.969488
Epoch 236
Validation binary_cross_entropy = 0.977890
Epoch 237
Validation binary_cross_entropy = 0.976210
Epoch 238
Validation binary_cross_entropy = 0.950947
Epoch 239
Loss = 3.0372e-02, PNorm = 99.1558, GNorm = 1.1906, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.925285
Epoch 240
Validation binary_cross_entropy = 0.923871
Epoch 241
Validation binary_cross_entropy = 0.930167
Epoch 242
Validation binary_cross_entropy = 0.952549
Epoch 243
Validation binary_cross_entropy = 0.965729
Epoch 244
Loss = 1.8484e-02, PNorm = 99.2777, GNorm = 0.8078, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.929674
Epoch 245
Validation binary_cross_entropy = 0.885682
Epoch 246
Validation binary_cross_entropy = 0.871371
Epoch 247
Validation binary_cross_entropy = 0.880756
Epoch 248
Validation binary_cross_entropy = 0.888276
Epoch 249
Loss = 2.2104e-02, PNorm = 99.3697, GNorm = 0.8191, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.899001
Epoch 250
Validation binary_cross_entropy = 0.947266
Epoch 251
Validation binary_cross_entropy = 0.998599
Epoch 252
Validation binary_cross_entropy = 1.042939
Epoch 253
Validation binary_cross_entropy = 1.034015
Epoch 254
Loss = 1.6638e-02, PNorm = 99.4766, GNorm = 0.0221, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.041932
Epoch 255
Validation binary_cross_entropy = 1.054870
Epoch 256
Validation binary_cross_entropy = 1.042200
Epoch 257
Validation binary_cross_entropy = 1.019178
Epoch 258
Validation binary_cross_entropy = 1.033964
Epoch 259
Loss = 9.6001e-03, PNorm = 99.5852, GNorm = 0.1387, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.069305
Epoch 260
Validation binary_cross_entropy = 1.015301
Epoch 261
Validation binary_cross_entropy = 0.965709
Epoch 262
Validation binary_cross_entropy = 0.928665
Epoch 263
Validation binary_cross_entropy = 0.916862
Epoch 264
Loss = 5.4796e-02, PNorm = 99.7578, GNorm = 1.4070, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.925097
Epoch 265
Validation binary_cross_entropy = 0.970365
Epoch 266
Validation binary_cross_entropy = 0.995877
Epoch 267
Validation binary_cross_entropy = 0.941317
Epoch 268
Validation binary_cross_entropy = 0.906935
Epoch 269
Loss = 2.9066e-03, PNorm = 99.9024, GNorm = 0.1128, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.881761
Epoch 270
Validation binary_cross_entropy = 0.877483
Epoch 271
Validation binary_cross_entropy = 0.875832
Epoch 272
Validation binary_cross_entropy = 0.892111
Epoch 273
Validation binary_cross_entropy = 0.914364
Epoch 274
Loss = 6.2037e-02, PNorm = 100.0374, GNorm = 0.1763, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.936746
Epoch 275
Validation binary_cross_entropy = 0.947211
Epoch 276
Validation binary_cross_entropy = 0.939925
Epoch 277
Validation binary_cross_entropy = 0.912028
Epoch 278
Validation binary_cross_entropy = 0.891237
Epoch 279
Loss = 3.7643e-02, PNorm = 100.1492, GNorm = 0.7130, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.911217
Epoch 280
Validation binary_cross_entropy = 0.947498
Epoch 281
Validation binary_cross_entropy = 0.999947
Epoch 282
Validation binary_cross_entropy = 1.050718
Epoch 283
Validation binary_cross_entropy = 1.080325
Epoch 284
Loss = 6.4289e-03, PNorm = 100.2718, GNorm = 0.0463, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.088692
Epoch 285
Validation binary_cross_entropy = 1.083258
Epoch 286
Validation binary_cross_entropy = 1.095780
Epoch 287
Validation binary_cross_entropy = 1.124130
Epoch 288
Validation binary_cross_entropy = 1.132845
Epoch 289
Loss = 4.7721e-03, PNorm = 100.3648, GNorm = 0.1830, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.668625
Epoch 290
Validation binary_cross_entropy = 1.131567
Epoch 291
Validation binary_cross_entropy = 1.101287
Epoch 292
Validation binary_cross_entropy = 1.063559
Epoch 293
Validation binary_cross_entropy = 1.034512
Epoch 294
Loss = 5.2607e-02, PNorm = 100.4282, GNorm = 0.9401, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.019683
Epoch 295
Validation binary_cross_entropy = 1.004948
Epoch 296
Validation binary_cross_entropy = 0.986524
Epoch 297
Validation binary_cross_entropy = 0.957911
Epoch 298
Validation binary_cross_entropy = 0.927415
Epoch 299
Loss = 5.9886e-03, PNorm = 100.5032, GNorm = 0.2469, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.900806
Model 0 best validation binary_cross_entropy = 0.247829 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.172683
Ensemble test binary_cross_entropy = 0.172683
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2400, bias=False)
        (W_h): Linear(in_features=2400, out_features=2400, bias=False)
        (W_o): Linear(in_features=2533, out_features=2400, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=2700, out_features=2400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=2400, out_features=1, bias=True)
  )
)
Number of parameters = 18,679,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.424301
Epoch 1
Validation binary_cross_entropy = 0.781818
Epoch 2
Validation binary_cross_entropy = 0.290956
Epoch 3
Validation binary_cross_entropy = 0.703059
Epoch 4
Loss = 5.5329e-01, PNorm = 87.7924, GNorm = 2.9020, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.278677
Epoch 5
Validation binary_cross_entropy = 0.461946
Epoch 6
Validation binary_cross_entropy = 0.458940
Epoch 7
Validation binary_cross_entropy = 0.298578
Epoch 8
Validation binary_cross_entropy = 0.431820
Epoch 9
Loss = 3.7779e-01, PNorm = 88.1913, GNorm = 3.0564, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.564251
Epoch 10
Validation binary_cross_entropy = 0.290464
Epoch 11
Validation binary_cross_entropy = 0.308254
Epoch 12
Validation binary_cross_entropy = 0.457934
Epoch 13
Validation binary_cross_entropy = 0.360325
Epoch 14
Loss = 4.0472e-01, PNorm = 88.5853, GNorm = 3.4729, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.297420
Epoch 15
Validation binary_cross_entropy = 0.332314
Epoch 16
Validation binary_cross_entropy = 0.323048
Epoch 17
Validation binary_cross_entropy = 0.329234
Epoch 18
Validation binary_cross_entropy = 0.365601
Epoch 19
Loss = 3.1902e-01, PNorm = 88.8546, GNorm = 1.8910, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.385806
Epoch 20
Validation binary_cross_entropy = 0.319524
Epoch 21
Validation binary_cross_entropy = 0.326340
Epoch 22
Validation binary_cross_entropy = 0.377455
Epoch 23
Validation binary_cross_entropy = 0.352016
Epoch 24
Loss = 2.5110e-01, PNorm = 89.0666, GNorm = 1.7048, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.308895
Epoch 25
Validation binary_cross_entropy = 0.342271
Epoch 26
Validation binary_cross_entropy = 0.446672
Epoch 27
Validation binary_cross_entropy = 0.341532
Epoch 28
Validation binary_cross_entropy = 0.321416
Epoch 29
Loss = 2.2067e-01, PNorm = 89.2721, GNorm = 1.5257, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.421479
Epoch 30
Validation binary_cross_entropy = 0.382771
Epoch 31
Validation binary_cross_entropy = 0.339294
Epoch 32
Validation binary_cross_entropy = 0.361539
Epoch 33
Validation binary_cross_entropy = 0.441804
Epoch 34
Loss = 2.1264e-01, PNorm = 89.4842, GNorm = 3.3074, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.446154
Epoch 35
Validation binary_cross_entropy = 0.366385
Epoch 36
Validation binary_cross_entropy = 0.367955
Epoch 37
Validation binary_cross_entropy = 0.371214
Epoch 38
Validation binary_cross_entropy = 0.477091
Epoch 39
Loss = 3.1712e-01, PNorm = 89.7194, GNorm = 1.4990, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.352124
Epoch 40
Validation binary_cross_entropy = 0.332439
Epoch 41
Validation binary_cross_entropy = 0.362520
Epoch 42
Validation binary_cross_entropy = 0.462692
Epoch 43
Validation binary_cross_entropy = 0.377754
Epoch 44
Loss = 2.0288e-01, PNorm = 89.9615, GNorm = 1.5064, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.343531
Epoch 45
Validation binary_cross_entropy = 0.350819
Epoch 46
Validation binary_cross_entropy = 0.385148
Epoch 47
Validation binary_cross_entropy = 0.486089
Epoch 48
Validation binary_cross_entropy = 0.488158
Epoch 49
Loss = 2.0635e-01, PNorm = 90.1834, GNorm = 1.0133, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.392796
Epoch 50
Validation binary_cross_entropy = 0.400210
Epoch 51
Validation binary_cross_entropy = 0.412613
Epoch 52
Validation binary_cross_entropy = 0.430464
Epoch 53
Validation binary_cross_entropy = 0.457363
Epoch 54
Loss = 1.3165e-01, PNorm = 90.4159, GNorm = 1.7751, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.441044
Epoch 55
Validation binary_cross_entropy = 0.407531
Epoch 56
Validation binary_cross_entropy = 0.403399
Epoch 57
Validation binary_cross_entropy = 0.394458
Epoch 58
Validation binary_cross_entropy = 0.420980
Epoch 59
Loss = 1.4094e-01, PNorm = 90.6533, GNorm = 1.5951, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.445704
Epoch 60
Validation binary_cross_entropy = 0.389465
Epoch 61
Validation binary_cross_entropy = 0.398226
Epoch 62
Validation binary_cross_entropy = 0.415671
Epoch 63
Validation binary_cross_entropy = 0.418549
Epoch 64
Loss = 1.5615e-01, PNorm = 90.9329, GNorm = 0.9165, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.422946
Epoch 65
Validation binary_cross_entropy = 0.425133
Epoch 66
Validation binary_cross_entropy = 0.426391
Epoch 67
Validation binary_cross_entropy = 0.428778
Epoch 68
Validation binary_cross_entropy = 0.448656
Epoch 69
Loss = 1.0826e-01, PNorm = 91.1870, GNorm = 0.7914, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.469113
Epoch 70
Validation binary_cross_entropy = 0.475788
Epoch 71
Validation binary_cross_entropy = 0.477236
Epoch 72
Validation binary_cross_entropy = 0.471878
Epoch 73
Validation binary_cross_entropy = 0.466840
Epoch 74
Loss = 9.0461e-02, PNorm = 91.4279, GNorm = 1.0242, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.455724
Epoch 75
Validation binary_cross_entropy = 0.448223
Epoch 76
Validation binary_cross_entropy = 0.445044
Epoch 77
Validation binary_cross_entropy = 0.450942
Epoch 78
Validation binary_cross_entropy = 0.462586
Epoch 79
Loss = 1.2578e-01, PNorm = 91.7379, GNorm = 1.6907, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.466227
Epoch 80
Validation binary_cross_entropy = 0.474489
Epoch 81
Validation binary_cross_entropy = 0.496064
Epoch 82
Validation binary_cross_entropy = 0.509894
Epoch 83
Validation binary_cross_entropy = 0.521787
Epoch 84
Loss = 1.3433e-01, PNorm = 92.0049, GNorm = 1.0597, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.532763
Epoch 85
Validation binary_cross_entropy = 0.528538
Epoch 86
Validation binary_cross_entropy = 0.533127
Epoch 87
Validation binary_cross_entropy = 0.543327
Epoch 88
Validation binary_cross_entropy = 0.528240
Epoch 89
Loss = 8.5522e-02, PNorm = 92.2946, GNorm = 0.9737, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.516393
Epoch 90
Validation binary_cross_entropy = 0.511192
Epoch 91
Validation binary_cross_entropy = 0.541011
Epoch 92
Validation binary_cross_entropy = 0.561767
Epoch 93
Validation binary_cross_entropy = 0.548680
Epoch 94
Loss = 4.6976e-02, PNorm = 92.6547, GNorm = 0.6703, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.556944
Epoch 95
Validation binary_cross_entropy = 0.573564
Epoch 96
Validation binary_cross_entropy = 0.571191
Epoch 97
Validation binary_cross_entropy = 0.546343
Epoch 98
Validation binary_cross_entropy = 0.540641
Epoch 99
Loss = 8.4388e-02, PNorm = 92.9891, GNorm = 1.1484, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.554883
Epoch 100
Validation binary_cross_entropy = 0.560788
Epoch 101
Validation binary_cross_entropy = 0.531662
Epoch 102
Validation binary_cross_entropy = 0.525245
Epoch 103
Validation binary_cross_entropy = 0.538648
Epoch 104
Loss = 5.0053e-02, PNorm = 93.2824, GNorm = 0.9013, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.561321
Epoch 105
Validation binary_cross_entropy = 0.574639
Epoch 106
Validation binary_cross_entropy = 0.575208
Epoch 107
Validation binary_cross_entropy = 0.584700
Epoch 108
Validation binary_cross_entropy = 0.587759
Epoch 109
Loss = 3.1993e-02, PNorm = 93.5701, GNorm = 0.7372, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.600773
Epoch 110
Validation binary_cross_entropy = 0.641505
Epoch 111
Validation binary_cross_entropy = 0.629689
Epoch 112
Validation binary_cross_entropy = 0.631662
Epoch 113
Validation binary_cross_entropy = 0.652836
Epoch 114
Loss = 7.1326e-02, PNorm = 93.8455, GNorm = 1.1480, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.674618
Epoch 115
Validation binary_cross_entropy = 0.671210
Epoch 116
Validation binary_cross_entropy = 0.678348
Epoch 117
Validation binary_cross_entropy = 0.702032
Epoch 118
Validation binary_cross_entropy = 0.698152
Epoch 119
Loss = 5.8019e-02, PNorm = 94.1352, GNorm = 1.1810, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.700086
Epoch 120
Validation binary_cross_entropy = 0.686786
Epoch 121
Validation binary_cross_entropy = 0.672406
Epoch 122
Validation binary_cross_entropy = 0.698414
Epoch 123
Validation binary_cross_entropy = 0.696528
Epoch 124
Loss = 3.1296e-02, PNorm = 94.4493, GNorm = 1.7368, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.693735
Epoch 125
Validation binary_cross_entropy = 0.710357
Epoch 126
Validation binary_cross_entropy = 0.714401
Epoch 127
Validation binary_cross_entropy = 0.711485
Epoch 128
Validation binary_cross_entropy = 0.695532
Epoch 129
Loss = 2.7422e-02, PNorm = 94.6778, GNorm = 1.0636, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.675987
Epoch 130
Validation binary_cross_entropy = 0.665188
Epoch 131
Validation binary_cross_entropy = 0.708377
Epoch 132
Validation binary_cross_entropy = 0.734638
Epoch 133
Validation binary_cross_entropy = 0.684294
Epoch 134
Loss = 3.8310e-02, PNorm = 94.9088, GNorm = 0.8215, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.670625
Epoch 135
Validation binary_cross_entropy = 0.673749
Epoch 136
Validation binary_cross_entropy = 0.685753
Epoch 137
Validation binary_cross_entropy = 0.681241
Epoch 138
Validation binary_cross_entropy = 0.695635
Epoch 139
Loss = 4.0401e-02, PNorm = 95.1319, GNorm = 0.7281, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.699694
Epoch 140
Validation binary_cross_entropy = 0.697754
Epoch 141
Validation binary_cross_entropy = 0.693414
Epoch 142
Validation binary_cross_entropy = 0.720855
Epoch 143
Validation binary_cross_entropy = 0.675967
Epoch 144
Loss = 2.2847e-02, PNorm = 95.3166, GNorm = 0.4927, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.620757
Epoch 145
Validation binary_cross_entropy = 0.620590
Epoch 146
Validation binary_cross_entropy = 0.629717
Epoch 147
Validation binary_cross_entropy = 0.653358
Epoch 148
Validation binary_cross_entropy = 0.685677
Epoch 149
Loss = 1.2308e-01, PNorm = 95.5298, GNorm = 0.7729, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.729034
Epoch 150
Validation binary_cross_entropy = 0.740378
Epoch 151
Validation binary_cross_entropy = 0.757521
Epoch 152
Validation binary_cross_entropy = 0.751315
Epoch 153
Validation binary_cross_entropy = 0.731943
Epoch 154
Loss = 7.4454e-02, PNorm = 95.7384, GNorm = 0.2050, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.721503
Epoch 155
Validation binary_cross_entropy = 0.745145
Epoch 156
Validation binary_cross_entropy = 0.735749
Epoch 157
Validation binary_cross_entropy = 0.722942
Epoch 158
Validation binary_cross_entropy = 0.734146
Epoch 159
Loss = 1.7054e-02, PNorm = 95.9508, GNorm = 0.7453, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.770469
Epoch 160
Validation binary_cross_entropy = 0.792265
Epoch 161
Validation binary_cross_entropy = 0.731226
Epoch 162
Validation binary_cross_entropy = 0.706337
Epoch 163
Validation binary_cross_entropy = 0.684109
Epoch 164
Loss = 3.7819e-02, PNorm = 96.1508, GNorm = 0.4734, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.689212
Epoch 165
Validation binary_cross_entropy = 0.689307
Epoch 166
Validation binary_cross_entropy = 0.684256
Epoch 167
Validation binary_cross_entropy = 0.680237
Epoch 168
Validation binary_cross_entropy = 0.683639
Epoch 169
Loss = 2.7924e-02, PNorm = 96.3506, GNorm = 0.8563, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.711742
Epoch 170
Validation binary_cross_entropy = 0.728241
Epoch 171
Validation binary_cross_entropy = 0.739055
Epoch 172
Validation binary_cross_entropy = 0.781215
Epoch 173
Validation binary_cross_entropy = 0.801634
Epoch 174
Loss = 1.2032e-01, PNorm = 96.5478, GNorm = 0.8071, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.777687
Epoch 175
Validation binary_cross_entropy = 0.756641
Epoch 176
Validation binary_cross_entropy = 0.759624
Epoch 177
Validation binary_cross_entropy = 0.790921
Epoch 178
Validation binary_cross_entropy = 0.812333
Epoch 179
Loss = 2.9444e-02, PNorm = 96.7698, GNorm = 0.7015, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.809327
Epoch 180
Validation binary_cross_entropy = 0.809232
Epoch 181
Validation binary_cross_entropy = 0.812343
Epoch 182
Validation binary_cross_entropy = 0.826406
Epoch 183
Validation binary_cross_entropy = 0.848753
Epoch 184
Loss = 4.1435e-02, PNorm = 96.9756, GNorm = 0.7432, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.835100
Epoch 185
Validation binary_cross_entropy = 0.825956
Epoch 186
Validation binary_cross_entropy = 0.834880
Epoch 187
Validation binary_cross_entropy = 0.895047
Epoch 188
Validation binary_cross_entropy = 0.951247
Epoch 189
Loss = 1.5517e-02, PNorm = 97.1983, GNorm = 0.2783, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.978139
Epoch 190
Validation binary_cross_entropy = 0.966308
Epoch 191
Validation binary_cross_entropy = 0.934576
Epoch 192
Validation binary_cross_entropy = 0.899038
Epoch 193
Validation binary_cross_entropy = 0.860058
Epoch 194
Loss = 4.0446e-02, PNorm = 97.3665, GNorm = 0.7536, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.817943
Epoch 195
Validation binary_cross_entropy = 0.801987
Epoch 196
Validation binary_cross_entropy = 0.789252
Epoch 197
Validation binary_cross_entropy = 0.793124
Epoch 198
Validation binary_cross_entropy = 0.796492
Epoch 199
Loss = 2.4813e-02, PNorm = 97.5465, GNorm = 0.8112, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.803072
Epoch 200
Validation binary_cross_entropy = 0.823132
Epoch 201
Validation binary_cross_entropy = 0.848074
Epoch 202
Validation binary_cross_entropy = 0.871552
Epoch 203
Validation binary_cross_entropy = 0.874981
Epoch 204
Loss = 3.7549e-02, PNorm = 97.6912, GNorm = 0.9345, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.896855
Epoch 205
Validation binary_cross_entropy = 0.910153
Epoch 206
Validation binary_cross_entropy = 0.898515
Epoch 207
Validation binary_cross_entropy = 0.877474
Epoch 208
Validation binary_cross_entropy = 0.884028
Epoch 209
Loss = 9.7885e-03, PNorm = 97.8165, GNorm = 0.2580, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.893800
Epoch 210
Validation binary_cross_entropy = 0.900687
Epoch 211
Validation binary_cross_entropy = 0.908382
Epoch 212
Validation binary_cross_entropy = 0.925882
Epoch 213
Validation binary_cross_entropy = 0.953515
Epoch 214
Loss = 3.8790e-02, PNorm = 97.9407, GNorm = 0.1662, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.974091
Epoch 215
Validation binary_cross_entropy = 1.002197
Epoch 216
Validation binary_cross_entropy = 1.049280
Epoch 217
Validation binary_cross_entropy = 0.974455
Epoch 218
Validation binary_cross_entropy = 0.888617
Epoch 219
Loss = 1.6957e-02, PNorm = 98.0504, GNorm = 0.0343, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.861337
Epoch 220
Validation binary_cross_entropy = 0.858927
Epoch 221
Validation binary_cross_entropy = 0.870242
Epoch 222
Validation binary_cross_entropy = 0.891569
Epoch 223
Validation binary_cross_entropy = 0.871650
Epoch 224
Loss = 3.9837e-02, PNorm = 98.1962, GNorm = 0.7460, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.892246
Epoch 225
Validation binary_cross_entropy = 0.933749
Epoch 226
Validation binary_cross_entropy = 0.928198
Epoch 227
Validation binary_cross_entropy = 0.892581
Epoch 228
Validation binary_cross_entropy = 0.853330
Epoch 229
Loss = 2.9758e-02, PNorm = 98.4071, GNorm = 0.8435, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.861656
Epoch 230
Validation binary_cross_entropy = 0.878926
Epoch 231
Validation binary_cross_entropy = 0.895001
Epoch 232
Validation binary_cross_entropy = 0.859304
Epoch 233
Validation binary_cross_entropy = 0.822306
Epoch 234
Loss = 1.5149e-02, PNorm = 98.5887, GNorm = 0.4252, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.797588
Epoch 235
Validation binary_cross_entropy = 0.797450
Epoch 236
Validation binary_cross_entropy = 0.833432
Epoch 237
Validation binary_cross_entropy = 0.873839
Epoch 238
Validation binary_cross_entropy = 0.917779
Epoch 239
Loss = 2.0541e-02, PNorm = 98.7484, GNorm = 0.7796, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.946990
Epoch 240
Validation binary_cross_entropy = 0.956382
Epoch 241
Validation binary_cross_entropy = 0.964981
Epoch 242
Validation binary_cross_entropy = 0.968452
Epoch 243
Validation binary_cross_entropy = 0.980761
Epoch 244
Loss = 5.9889e-03, PNorm = 98.8888, GNorm = 0.1822, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.993304
Epoch 245
Validation binary_cross_entropy = 0.972021
Epoch 246
Validation binary_cross_entropy = 0.935993
Epoch 247
Validation binary_cross_entropy = 0.928480
Epoch 248
Validation binary_cross_entropy = 0.927223
Epoch 249
Loss = 1.1243e-02, PNorm = 99.0150, GNorm = 0.3395, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.951402
Epoch 250
Validation binary_cross_entropy = 0.981753
Epoch 251
Validation binary_cross_entropy = 0.932495
Epoch 252
Validation binary_cross_entropy = 0.921816
Epoch 253
Validation binary_cross_entropy = 0.917658
Epoch 254
Loss = 2.4754e-02, PNorm = 99.1964, GNorm = 0.3277, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.928873
Epoch 255
Validation binary_cross_entropy = 0.925103
Epoch 256
Validation binary_cross_entropy = 0.902740
Epoch 257
Validation binary_cross_entropy = 0.900738
Epoch 258
Validation binary_cross_entropy = 0.924416
Epoch 259
Loss = 6.9008e-03, PNorm = 99.3877, GNorm = 0.1786, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.961584
Epoch 260
Validation binary_cross_entropy = 0.951312
Epoch 261
Validation binary_cross_entropy = 0.948288
Epoch 262
Validation binary_cross_entropy = 0.949033
Epoch 263
Validation binary_cross_entropy = 0.931714
Epoch 264
Loss = 1.4572e-02, PNorm = 99.5210, GNorm = 0.5104, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.896323
Epoch 265
Validation binary_cross_entropy = 0.905132
Epoch 266
Validation binary_cross_entropy = 0.912178
Epoch 267
Validation binary_cross_entropy = 0.943638
Epoch 268
Validation binary_cross_entropy = 0.963111
Epoch 269
Loss = 8.2976e-03, PNorm = 99.6485, GNorm = 0.1971, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.969515
Epoch 270
Validation binary_cross_entropy = 0.959443
Epoch 271
Validation binary_cross_entropy = 0.930793
Epoch 272
Validation binary_cross_entropy = 0.900137
Epoch 273
Validation binary_cross_entropy = 0.885854
Epoch 274
Loss = 1.2746e-02, PNorm = 99.7665, GNorm = 0.3181, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.872391
Epoch 275
Validation binary_cross_entropy = 0.863789
Epoch 276
Validation binary_cross_entropy = 0.859946
Epoch 277
Validation binary_cross_entropy = 0.848861
Epoch 278
Validation binary_cross_entropy = 0.846711
Epoch 279
Loss = 7.4547e-03, PNorm = 99.8820, GNorm = 0.2710, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.855670
Epoch 280
Validation binary_cross_entropy = 0.864522
Epoch 281
Validation binary_cross_entropy = 0.886462
Epoch 282
Validation binary_cross_entropy = 0.912521
Epoch 283
Validation binary_cross_entropy = 0.947909
Epoch 284
Loss = 6.6617e-02, PNorm = 99.9856, GNorm = 0.5863, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.982287
Epoch 285
Validation binary_cross_entropy = 1.028154
Epoch 286
Validation binary_cross_entropy = 1.024165
Epoch 287
Validation binary_cross_entropy = 0.970898
Epoch 288
Validation binary_cross_entropy = 0.924690
Epoch 289
Loss = 4.5832e-03, PNorm = 100.1296, GNorm = 0.2228, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.904614
Epoch 290
Validation binary_cross_entropy = 0.894392
Epoch 291
Validation binary_cross_entropy = 0.878531
Epoch 292
Validation binary_cross_entropy = 0.875470
Epoch 293
Validation binary_cross_entropy = 0.892738
Epoch 294
Loss = 1.3590e-02, PNorm = 100.2661, GNorm = 0.1617, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.937323
Epoch 295
Validation binary_cross_entropy = 0.977998
Epoch 296
Validation binary_cross_entropy = 0.995542
Epoch 297
Validation binary_cross_entropy = 0.974631
Epoch 298
Validation binary_cross_entropy = 0.942484
Epoch 299
Loss = 6.6040e-03, PNorm = 100.3954, GNorm = 0.2814, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.935200
Model 0 best validation binary_cross_entropy = 0.278677 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.224669
Ensemble test binary_cross_entropy = 0.224669
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.173575
	Seed 1 ==> test binary_cross_entropy = 0.193772
	Seed 2 ==> test binary_cross_entropy = 0.218211
	Seed 3 ==> test binary_cross_entropy = 0.245994
	Seed 4 ==> test binary_cross_entropy = 0.161298
	Seed 5 ==> test binary_cross_entropy = 0.237489
	Seed 6 ==> test binary_cross_entropy = 0.184349
	Seed 7 ==> test binary_cross_entropy = 0.244032
	Seed 8 ==> test binary_cross_entropy = 0.172683
	Seed 9 ==> test binary_cross_entropy = 0.224669
Overall test binary_cross_entropy = 0.205607 +/- 0.030515
Elapsed time = 0:11:28
